aussie bloke

Xcode 4.1 and 100% CPU usage

2011-12-09T23:23:00.001-07:00

Still needing to use Xcode 4.1 and find that it slowly consumes 100% CPU over time? Turns out there is a nasty bug in Xcode when you have WiFi sync enabled on iOS 5.x devices; Xcode continually tries to connect to these devices running the newer iOS and fails. Abysmally.

All is not lost, and if you install SIMBL, you can use a nifty plugin by evands on GitHub. It only loads itself in Xcode versions ≤ 4.1, and works around the issue.

Constant Confusion

2011-12-03T01:33:00.001-07:00

The const keyword in C can be confusing, so I wanted to put down my thoughts for my own benefit and for those looking for some clarity. In summary, I'm going to advocate you place const to the right, and read the declaration from right to left.

What is const?

const is a hint to the C compiler and programmer that a specific declaration or element of a declaration is immutable. It is more complicated in C++, which is outside the scope of this post. Lets start with a simple example, and the most common form where const is written first:

const int i = 5;

The previous statement simply instructs the compiler that i cannot be changed after its initial declaration, such that the following assignment would result in an error:

const int i = 5;
i = 6; // error: read-only variable is not assignable

If spoken from right to left, the declaration of i would read “i is an integer constant”, which is reasonable enough. This syntax is still supported for historical reasons. The alternative and correct format is to write const after the type reference, as follows

int const i = 5;
i = 6; // error: read-only variable is not assignable

which means the same thing; the value of i must remain constant within its declared scope. If you read this declaration from right to left, it is spoken as “i is a constant integer”; much better.

Yet another common definition is an array of constant characters or also known in some circles as an immutable string

const char * myString = "hello World";
myString[0] = 'H';        // error: read-only variable is not assignable
myString = "Hello World"; // ok

Spoken from right to left, “myString is a pointer to character constant”, which doesn't read so well. Switching the const keyword, we get the following

char const * myString = "hello World";
myString[0] = 'H';        // error: read-only variable is not assignable
myString = "Hello World"; // ok

read aloud, it is “myString is a pointer to constant characters“; sounds better than the former.

So far const seems pretty straight forward, and perhaps at this point you're thinking where is the confusion?. Lets complicate things… As we've demonstrated, the const keyword can be placed before or after the type reference in a declaration, which for the previous scenarios is fine. Let's show a few more examples with pointers:

int * const i;

Declares i as constant pointer to an integer. Again, reading from right to left makes it pretty clear what we're dealing with. The following shows both legal and illegal usages of i:

int j = 5, k = 6;
int * const i = &j; // points to the memory referred to by j
i = &k;             // error: read-only variable is not assignable
*i = 6;             // ok, modifies the value of j

Now, because the const keyword can be placed before or after the type reference, as shown earlier, the following declarations of h and i:

int j = 5, k = 6;
const int * const h = &j;
int const * const i = &j;

result in a variable that is a constant pointer to a constant integer. Given that, the following lines will result in compiler errors:

*h = 5; // error: read-only variable is not assignable
h = &k; // error: read-only variable is not assignable

Want to see something confusing? The following 3 lines are identical, and will compile with current versions of clang and gcc:

const int const g = 5;
const int h = 6;
int const i = 7;

It's not horrible, but lets reintroduce pointers

const int const * i;

What is this? If you had not just read all the above, one would be forgiven for thinking i is a constant pointer to a constant integer, but in fact both instances of const refer to the int type; this is a pointer to a constant integer. The correct definition, as we saw earlier is

const int * const i; // or preferably
int const * const i;

Worse still is multiple levels of indirection, such as

int * * i;

which when read from right to left declares “i is a pointer to a pointer of integers”. A more complicated example using the const keyword

int const * const * const i;

or “i is a constant pointer to a constant pointer of constant integers. Wow. All levels of indirection are immutable, so it can only be assigned at its declaration.

int j[2][3] = { {1, 2, 3}, {4, 5, 6} };
int const * const * const i = &j; // ok
i = NULL;     // error
i[0] = &j[0]; // error
i[0][0] = 5;  // error

It's an extreme case, but becomes evident why writing const to the right is a good habit. The following declaration of i may look the same to the untrained eye:

int j[2][3] = { {1, 2, 3}, {4, 5, 6} };
const int const * const * i = &j; // ok
i = NULL;     // ok!!
i[0] = &j[0]; // error
i[0][0] = 5;  // error

however, armed with this knowledge, you (and the compiler) say otherwise.

After all this, you may be asking when might I use the declaration type const * const? When possible, I prefer to be explicit, and strings used as constants across compilation units are often declared as

char const * kMyConstant = "Hello World";

and then referred to in a separate object file as

extern char const * kMyConstant;

Whilst modifying the contents of kMyConstant is prohibited, assigning the pointer kMyConstant to a new location is possible, as follows

extern char const * kMyConstant;
void fn() {
    kMyConstant[0] = 'h'; // error
    kMyConstant = "Hi!";  // now says "Hi!" instead of "Hello World"
}

Clearly for the majority of situations, this is not what the developer intended. Changing the declaration to

char const * const kMyConstant = "Hello World";

would also prevent the pointer kMyConstant from being overwritten.

In summary

Always write const to the right and read the declaration from right to left.

Google Maps WebGL with Safari 5.1

2011-10-13T15:28:00.001-07:00

It's possible to use the WebGL version of Google Maps with Safari, if you are willing to roll up your sleeves a little. Follow these steps to try it out:

Enable the Develop menu by opening Preferences and selecting the Advanced tab; check the Show Develop menu in menu bar option
In the newly enabled Develop menu, enable WebGL:
Set your UserAgent to Other, which is available in the Develop menu and paste in the following user agent string (which will fool Google maps into thinking your browser is Chrome):
- ```
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.92 Safari/535.2
```
Navigate to http://maps.google.com/gl
- Down the bottom left you will see an option to try Google Maps with WebGL

Using your iOS 5 device with Xcode 4.1

2011-09-09T22:17:00.001-07:00

To enable iOS 5 support in Xcode 4.1, read on.

You've upgraded one or more devices to iOS 5.0 and try to use Xcode 4.1, only to be told it doesn't recognize the version of the OS. Why would you care? Apple has been advising for some time that gcc is going away, and that date may be fast approaching. Unfortunately, Clang / llvm lack some features available in gcc, such as register assigned local or global variables. We've only just gained access to a version of Clang that supports C++, and so you may still need some time to migrate your code.

Assuming Xcode 4.1 is in /Developer and Xcode 4.2 is in /Xcode42, run the following, substituting the appropriate iOS 5 build numbers:

cd /Developer/Platforms/iPhoneOS.platform/DeviceSupport
ln -s /Xcode42/Platforms/iPhoneOS.platform/DeviceSupport/5.0\ \(9A5313e\)/ 5.0\ \(9A5213e\)

iCloud Key-Value is better than a UDID

2011-08-19T23:26:00.001-07:00

There is a lot of chatter about Apple depreciating the UDID API and apparently providing no alternative to track a user. Firstly, lets clear up how insecure and unreliable using the UDID is for identifying a user, which is tied to the device. The user's device could be

lost,
stolen,
replaced / exchanged,
sold or
given away.

That UDID is now potentially in the hands of another individual, even if you remote wipe your device.

Beginning with iOS 5, Apple is providing a secure service called iCloud Key-Value storage that will serve as a superior alternative. iCloud K-V storage offers a limited amount of space in the cloud for a single application or shared between a suite of applications; either scenario is at the app developer's discretion. If your application requires access to it's own servers for user-specific information, you can store a unique token in their iCloud K-V store to reunite your application with their data, without them ever having to log in again* after the initial handshake. This applies even when the user replaces their device, as the token is stored in the cloud.

Apparently, Pandora uses the UDID to remember your profile and potentially exposes your playlists if your device falls victim to one of the more dubious fates mentioned above. Easily solved if you follow a pattern similar to the following sequence diagram:

Naturally, you should take the appropriate measures to secure the conversation between your app and your server.

If you do need to track a specific device, there are ways to programmatically obtain the MAC address. Not something I would recommend.

* Apple is yet to address a significant caveat regarding the lack of multiple Apple IDs under a single iTunes account. Your iCloud account is your Apple ID, and that implies all your family members must log in to the same account, providing no way to separate users.

Local (file system) development with Google Chrome

2011-08-05T17:06:00.001-07:00

If you use Google Chrome for an advanced local file system development on Mac OS X, no doubt you've been frustrated with issues such as security (same origin). All is not lost, as I've created an application bundle that allows you to launch Chrome from Finder with the following additional command-line switches:

disable-web-security
- Don't enforce the same-origin policy. (Used by people testing their sites.)
- Example: http://learnwebgl.com lessons; much better than having to set up apache

allow-file-access-from-files
- By default, file:// URIs cannot read other file:// URIs. This is an override for developers who need the old behavior for testing.

You can download the archive from here.

If there are other switches you wish to add, many of which are conveniently documented here,

RMB and select Show Package Contents and
edit the shell script in Google Chrome Dev.app/Contents/MacOS

Transferring Preview app signatures in Lion

2011-08-03T22:45:00.001-07:00

Lion introduced a great new feature that allows you to capture your signature via an attached camera and store it in an encrypted form for later use. Therein lies the problem; you must have an attached camera.

I have a Mac Pro, and wanted to use the signatures I captured on my Macbook Pro. Following these steps, you can transfer the encrypted signatures over.

On your machine endowed with the power of sight:

Go ahead and capture the signatures in Preview.
Quit Preview
Browse to ~/Library/Containers/com.apple.Preview/Data/Library/Preferences
- In Finder, click the Go menu and hold the option (⌥) key to show the Library folder, alternatively
- press ⌘+⇧+G whilst Finder is active and enter the path above to directly navigate
Copy the com.apple.Preview.signatures.plist
Launch Keychain Access
Ensure the login keychain is selected and the Passwords category
Right-click the Preview Signature Privacy password and select Copy Password to Clipboard. This is the password used to encrypt the signature images.
Paste it into a text editor and save the file. You'll need to transfer this to your other computer(s)

On your machine you wish to transfer the signatures to:

If you haven't already, launch Preview, open preferences, and select the Signatures tab to ensure the default configuration files and folders have been created. Exit Preview.
Browse to ~/Library/Containers/com.apple.Preview/Data/Library/Preferences
Copy the com.apple.Preview.signatures.plist to the folder, overwriting any existing file
Launch Keychain Access
Locate the Preview Signature Privacy password in the login keychain and double click to edit
Click the Show password checkbox and paste the password you copied from your original machine.
Click Save Changes and you're done

Eval Expression service for OS X

2011-04-09T21:44:00.001-07:00

Eval Expression is a Mac OS X service to evaluate the selected text of any text field as a Ruby expression. My instinct was to choose Perl, however Ruby offers binary in addition to decimal, hex and octal numerical literals. The service becomes infinitely more useful if you assign it a global shortcut in Keyboard preferences. In my case I assigned a combination that seemed obvious, ⌘=

It came about as I was working on some layout in Xcode 4 / Interface Builder, and needed to adjust the Y position of a view by a specific number of units. I'm lazy, and figured the computer should do the work, so I typed 768-35 and tabbed out. Unlike Acorn and Opacity, which evaluates the expression in numerical entries automatically, Xcode simply complained. Not happy, I created this service and it works like a charm. Some examples:

768-35 ⌘= 733
- calculator
0b11010 ⌘= 26
- conversion to decimal
0x1f.to_s(2) ⌘= 11111
- hex to binary
2**3 ⌘= 8
- exponent
(0x8000 | 0xf1).to_s(16) ⌘= 80f1
"-+" * 5 ⌘= -+-+-+-+-+
- expansion
Array(1..15).join ',' ⌘= 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
- lists

Download the Automator zip file from github, unzip and install to your ~/Library/Services folder.

A quick video of the service in action (note that I'm using my ⌘= shortcut to evaluate within Mars Edit):

Mobile Safari performance and executable pages in iOS 4.3

2011-03-17T15:12:00.001-07:00

DaringFireball recently posted his thoughts as to why the Javascript performance of Mobile Safari is faster than those launched from SpringBoard or within existing applications which use a UIWebView. I don't think it's quite that complicated.

My thoughts are it is one of two things:

Apple is concerned about security (which would support John's theory)
- Mobile Safari is binding to a newer and/or private version of Webkit, and Springboard was an oversight
Apple is concerned about breaking existing applications
- I would argue this makes the most sense, and was probably a very deliberate action by Apple, given enabling Nitro is a significant enough change to a core framework that could break existing applications.
- Perhaps a future update will allow developers to opt-in to the faster Javascript engine either via a plist setting or more likely, a new API on the UIWebView class.
- Would also explain the oversight in Springboard, if it was not opted in.

There are other implications for allowing 3rd parties to use Nitro, most importantly battery life. If Javascript executes up to 3x faster, the CPU is running 3x less to to execute equivalent Javascript, allowing the CPU to go to low power mode faster.

Gruber also makes the following statement:

Apple, as of iOS 4.3, trusts Mobile Safari enough to allow this. The upside is that Mobile Safari is now significantly faster. The downside is that any security exploits against Mobile Safari now potentially allow worse things to happen than before.

I would disagree, as many of the previous Jailbreak exploits in iOS such as this example were via Mobile Safari, without a JIT engine, simply exploiting the stack and other typical attacks to elevate privileges. As Apple continues to get a handle on these security issues, running dynamic code shouldn't be a problem.

What is my basis for this assumption?

A few weeks ago, when I heard Apple's announcement that Nitro was coming to 4.3, I immediately had to retest some scenarios that were not previously supported on iOS. Prior to iOS 4.3, calling the mprotect function and attempting to set the PROT_EXEC flag, marking the page as executable would fail when run on the device. Quite simply, the API would return an error code. This suggested that likely something in the kernel was locked down or this API had specific code to disallow pages to be marked as executable. Knowing that the Nitro engine must execute in user space or it would be a serious hack in iOS, Apple had to enable support for executable pages. I went ahead and created an example, ran it on my iPhone 4 and viola! Essentially this example creates an 'increment' function in 32-bit ARM on the fly that adds one to the parameter and returns the result.

I responded to Miguel de Icaza's tweet, who also wondered it might now be possible. Further discussions with Miguel also identified that Apple has only partially opened up executable page support, whereby pages can either be read/write or read/execute. Unfortunately the majority of existing JIT engines expect RWX in order to patch executable pages without the need to change the state from rx » rw, patch code and then rw » rx. Critical in multi-threaded scenarios, as code could conceivably jump to a page that was being patched and fault due to the absence of the execute flag.

Renew Apple developer certificates with OpenSSL

2011-02-02T16:30:00.001-07:00

I like to reuse the same private keys when generating a signing request to renew my Apple developer certificates. Unfortunately you can't do this with Keychain Access, as it won't save the signing request file after you step through the wizard. OpenSSL is your friend.

Open Keychain Access, RMB on the key your wish to use and click Export "[Key Name]". Save it as a .p12 file with a strong password. In my case it was StuartCarnie.p12.

Open a terminal session and convert the .p12 to a PEM with the following command

openssl pkcs12 -in StuartCarnie.p12 -out StuartCarnie.pem

You will be prompted for your .p12 password, and also a password to encrypt your .pem.

Now generate the signing request with the following command.

openssl req -new -key StuartCarnie.pem -out StuartCarnie.csr

You'll be prompted for a few questions to place in the signing request, such as the country, etc. At the very least, enter the Common Name (your name) and Email Address. One you've completed this step, the .csr file can be submitted to Apple.

iPhone Templates in Google Docs

2010-04-12T21:25:00.001-07:00

I have created a set of iPhone UI templates using Google Docs Drawings for sharing / collaborating layout ideas. You can access the template here. If you are logged in to your Google account, create a copy by selecting the File menu.

Micro-benchmarking 2nd / 3rd gen iPhones and iPad

2010-01-24T22:58:00.002-07:00

Update: Added iPhone 4, iPad 1^st gen.
I follow the excellent weekly posts by Mike Ash, and entered a brief discussion in comments about toll free bridging. In particular, the difference between calling a method via Objective-C (objc_msgSend) and it's equivalent CoreFoundation C call. Mike suggested adding it to his original suite of tests, which lead to the following results.

iPhone 4 (-fthumb)

Apple A4 ARM Cortex A8 ~800MHz

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	0.9	9.0
C++ virtual method call	100000000	1.0	10.2
16 byte memcpy	10000000	0.4	36.2
Integer division	100000000	4.1	40.6
Objective-C message send	100000000	4.1	40.8
Floating-point division	10000000	0.9	89.4
Objective-C objectAtIndex:	10000000	1.1	105.8
Float division with int conversion	10000000	1.1	105.8
CF CFArrayGetValueAtIndex	10000000	1.7	168.1
NSInvocation message send	100000	0.1	550.8
16 byte malloc/free	10000000	6.6	656.3
NSAutoreleasePool alloc/init/release	100000	0.1	979.5
NSObject alloc/init/release	100000	0.4	4277.9
16MB malloc/free	1000	0.0	20406.7
pthread create/join	100	0.0	139971.2
Zero-second delayed perform	1000	0.2	243883.3
1MB memcpy	100	0.1	1150657.9

iPhone 3GS (-fthumb)

ARM Cortex A8 ~600MHz / 1.66 ns per cycle

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	1.2	11.7
C++ virtual method call	100000000	1.3	13.5
16 byte memcpy	10000000	0.5	46.0
Objective-C message send	100000000	5.4	53.9
Integer division	100000000	6.3	62.9
Floating-point division	10000000	1.2	117.4
Float division with int conversion	10000000	1.4	138.2
Objective-C objectAtIndex:	10000000	1.4	140.1
CF CFArrayGetValueAtIndex	10000000	2.2	220.0
16 byte malloc/free	10000000	6.4	642.6
NSInvocation message send	100000	0.1	723.0
NSAutoreleasePool alloc/init/release	100000	0.1	1305.9
NSObject alloc/init/release	100000	0.6	5743.7
16MB malloc/free	1000	0.0	16104.0
pthread create/join	100	0.0	185759.2
Zero-second delayed perform	1000	0.4	353519.4
1MB memcpy	100	0.2	2170179.2

iPhone 3GS (no thumb)

ARM Cortex A8 ~600MHz / 1.66 ns per cycle

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	1.2	11.8
C++ virtual method call	100000000	4.3	42.9
Objective-C message send	100000000	5.9	59.2
CF CFArrayGetValueAtIndex	10000000	1.0	97.9
Integer division	100000000	9.8	98.4
16 byte memcpy	10000000	1.1	109.3
Floating-point division	10000000	1.2	118.5
Objective-C objectAtIndex:	10000000	1.3	129.0
Float division with int conversion	10000000	1.4	142.6
16 byte malloc/free	10000000	7.5	748.6
NSInvocation message send	100000	0.1	806.0
NSObject alloc/init/release	100000	0.5	4793.1
NSAutoreleasePool alloc/init/release	100000	0.5	4953.1
16MB malloc/free	1000	0.0	17969.2
Zero-second delayed perform	1000	0.2	211840.4
pthread create/join	100	0.0	214742.5
1MB memcpy	100	0.3	3162774.6

iPhone 3G

ARM1176 ~412MHz / 2.4ns per cycle

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	3.9	38.6
C++ virtual method call	100000000	5.0	49.9
Floating-point division	10000000	0.8	81.3
Float division with int conversion	10000000	0.8	81.4
16 byte memcpy	10000000	1.4	136.0
Objective-C message send	100000000	14.9	148.6
Integer division	100000000	16.2	162.2
CF CFArrayGetValueAtIndex	10000000	2.0	201.7
Objective-C objectAtIndex:	10000000	4.2	418.3
NSInvocation message send	100000	0.2	1833.2
16 byte malloc/free	10000000	27.3	2729.8
NSObject alloc/init/release	100000	1.4	14179.1
NSAutoreleasePool alloc/init/release	100000	1.9	18956.7
16MB malloc/free	1000	0.0	47811.3
Zero-second delayed perform	1000	0.8	803419.3
pthread create/join	100	0.1	1085830.0
1MB memcpy	100	1.0	9902796.7

iPad (-fthumb)

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	0.7	7.1
C++ virtual method call	100000000	0.8	8.1
16 byte memcpy	10000000	0.3	27.7
Objective-C message send	100000000	3.2	32.3
Integer division	100000000	3.4	33.7
CF CFArrayGetValueAtIndex	10000000	0.6	58.8
Floating-point division	10000000	0.7	70.5
Objective-C objectAtIndex:	10000000	0.8	81.6
Float division with int conversion	10000000	0.8	83.1
16 byte malloc/free	10000000	3.6	357.8
NSInvocation message send	100000	0.0	470.8
NSAutoreleasePool alloc/init/release	100000	0.3	2957.0
NSObject alloc/init/release	100000	0.3	3080.2
16MB malloc/free	1000	0.0	14824.2
pthread create/join	100	0.0	127386.2
Zero-second delayed perform	1000	0.2	225271.3
1MB memcpy	100	0.1	1064566.2

iPad (-fno-thumb)

Apple A4 ARM Cortex A8 ~1GHz / 1 ns per cycle

Name	Iterations	Total time (sec)	Time per (ns)
IMP-cached message send	100000000	0.8	8.1
C++ virtual method call	100000000	2.2	21.8
16 byte memcpy	10000000	0.3	28.2
Objective-C message send	100000000	3.2	32.5
Integer division	100000000	3.4	33.9
CF CFArrayGetValueAtIndex	10000000	0.6	55.8
Floating-point division	10000000	0.7	70.9
Objective-C objectAtIndex:	10000000	0.8	81.6
Float division with int conversion	10000000	0.8	82.8
16 byte malloc/free	10000000	3.6	358.3
NSInvocation message send	100000	0.0	473.4
NSAutoreleasePool alloc/init/release	100000	0.3	3017.6
NSObject alloc/init/release	100000	0.3	3071.8
16MB malloc/free	1000	0.0	14623.6
pthread create/join	100	0.0	128674.6
Zero-second delayed perform	1000	0.3	255627.5
1MB memcpy	100	0.1	1063407.5

Note that I did reduce the iterations from the original tests, so whilst the total times are significantly less, the iteration times are still a reflection of overall performance. Compared to Mike's results, these show that the IMP method is indeed faster as expected, but this was only after I changed to a release build. I also compiled these with Thumb disabled unless otherwise specified.
I've recently watched some iTunes U videos released by Apple on optimizing OpenGL ES 2.0 and a key takeaway was that the Cortext A8 architecture should always be compiled with thumb enabled. The Cortex CPU uses the newer Thumb-2 instruction set, which has native instructions for floating point. The benefit of Thumb is reduced code size and potentially better performance by utilising the I-cache.

Observations

I've estimated the iPhone 4 CPU to be running at 800MHz. Looking at the increased speed over the 3GS for a number of benchmarks, the average is 1.333x increase. Multiplying 600MHz x 1.333 yields roughly 800MHz as the clock speed.
The IMP-cached message send is significantly faster on the newer Cortex CPU. I have read of improvements in the branch prediction logic, which is particularly important due to the greater penalty of a misprediction in the longer A8 pipeline. The code for executing the call is
blx r8
r8 contains the target address of the function, and remains so for the duration of the test.
For the 3GS, the Objective-C message send is very close to the C++ virtual method call. I ran this test several times, and the behaviour didn't change. The virtual method call is an indirect load of the pc register
ldr pc, [r3]
Without being able to access the PMC registers, I can't be sure of mispredictions; however, I know that 9 instructions are executed every iteration in the C++ test. That suggests around 15ns / iteration; but, we're at 42.9. Adding an additional 13 cycles every iteration (21.58ns) for a mispredition would get us to 37ns / iteration - much closer. Stepping in to the objc_msgSend function finds the cached method on the first pass, totaling 28 instructions per iteration. Given there are significantly more instructions for the Objective-C call, we're probably seeing the benefits of the dual—issue architecture.
Memory performance of the 3GS is significantly higher. I've done some other micro-benchmarks, showing 2^nd gen around 200 MB/s and 3^rd gen around 800MB/s. With some very well placed cache-preloads, I've actually pushed the ARM1176 to almost 300MB/s.
Calling the objectAtIndex: using CoreFoundation API is 2x faster on older devices; however, the gap is less significant with the newer hardware. We've seen significant improvements to the objc_msgSend performance on the 3GS, which undoubtedly is making up much of the gap.
Floating point performance for scalar operations is slightly slower on the newer device.

Source code for this test is available here.

C64 for iPhone: Rejected.

2009-06-20T02:50:00.000-07:00

After a deafening silence, I can finally talk about my little side project. I posted my initial efforts about a year ago and after the excitement of TUAW publishing a story, I spent some more time adding a keyboard and improving the performance with dreams of an App Store release. That is, until reality sunk in. There is an incredible amount of work to turn a concept like this into a polished, user friendly and legal product, ready for sale. I attempted to find who owned the Commodore 64 brand, but constantly hit dead ends. I finally took a break from C64 and played around with new projects, like the SID player. It has made progress, but I'll leave that for another post.

Not much happened with the emulator for some time; however, everything changed when I received an email from Brian Lyscarz, a Danish entrepreneur who is now a resident of Sweden. It turns out he is just as passionate as I when it comes to retro, and had personally funded some initial development of a C64 emulator for the iPhone. Fortunately (for me) this didn't go too far and Brian found me because of the initial press. Aside from an initial phone call, we have communicated entirely via email, post and Google chat, to achieve what follows.

We concluded (having never met) that the next obvious step was to form a company dedicated to retro gaming, and Manomio LLC was born.

As the months have gone by, we've really settled into what has become a great partnership. Essentially, I got it working and Brian had the aesthetic eye and the skill with Photoshop to make it pretty:

The next hurdle was licensing. Fortunately, Brian knew the right people, which lead to Manomio securing an official license for the brand from Commodore Gaming and Kiloo Apc.

The final hurdle was Apple and the SDK agreement, section 3.2.2. We contacted Apple Developer Relations in the United Kingdom and explained our approach. In principal, we don't allow you to download arbitrary content - we'll secure the licenses and release game packs officially via the App Store. We agree it's not ideal, but we had to start somewhere. If Apple loosens the reins, so will we! They were very excited by what we had built and assured us we'd be okay given we weren't directly competing with the App Store and had locked down the emulator to only installing official titles. They also mentioned there were already other 'emulators' available, like the SID player and SC68, which actually can download content freely via the net. This leads us to today - we've been rejected on those very grounds. We're going to resubmit with no access to BASIC, so it simply plays games in the hope it will be perceived as just a pack of games.

Hopefully we can be noisy across the digital communication channels and perhaps Apple will change their mind. We have our first review in too, so go check it out!

Enough talk, here it is in action on a 3G (not my new 3Gs):

Enable -Wformat for better compile time help

2009-04-18T16:16:00.001-07:00

Let the compiler do all the hard work, and be sure to enable the following warning:

It does more than just validate printf/scanf formatting calls, which is helpful in itself. It also validates that a sentinel is present in variadic functions. A sentinel is typically NULL or nil for the last parameter. A common place you would benefit from this is using the arrayWithObjects method of NSArray, that requires a nil for the last value.

NSArray *items = [NSArray arrayWithObjects:@"one", @"two", nil];

If the nil is absent, you're receive the following:

Small Zip / Unzip library for iPhone

2009-04-03T09:08:00.001-07:00

I have looked with ZipArchive, which is an Objective-C compression framework; however, I ended up using Lite Zip / Unzip on codeproject as it is just two .C files. Include one for compressing ZIP files and the other for decompressing.

XCode Tip: Generate comments in your assembler output

2009-03-07T21:56:00.002-07:00

To make it easier to find the assembly generated when you 'Show Assembly Code', embed comments using:

asm("# your comment")

How to debug handleOpenURL

2009-01-30T23:33:00.041-07:00

I've seen a number of questions on the Apple iPhone developer forums asking how to debug the UIApplication handleOpenURL message. I finally had a complex scenario that I needed to debug, and came up with the following solution. Note that this example has been tested using the simulator only. Firstly, you'll need to grab the two 'DebugSupport' files from my google code repository here. Include them in your project and modify your handleOpenURL message as follows:

-(BOOL)application:(UIApplication *)application handleOpenURL:(NSURL *)url {
 [DebugSupport waitForDebugger];
 ...
 // place breakpoint after the above line
}

The call to [DebugSupport waitForDebugger] shows a UIAlertView, which will wait until you confirm by clicking the OK button. You'll notice that the prompt tells you the process ID. Don't click OK yet. Return to XCode and from the Run menu, choose Attach To Process | Process ID... Enter the PID given to you from the alert box and XCode will attach and enable all the breakpoints. Obviously, remove this from production code. Enjoy.

iPhone 3.0 Wish List

2009-01-30T09:03:00.009-07:00

I took this from a comment I made on TUAW

Here's what I'd like to see, assuming we'll also get iPhone 3.0 software update.

Hardware

The entire screen should be touch enabled, so the border could be used for interactivity.
ARM Cortex A9 (multi-core version would be a dream),
PowerVR SGX and
minimum of 256MB RAM (up from 128MB)

Software, given the above specs

Background applications

I'm wondering if this is coming rather than Apple's original 'push' solution, since we haven't seen it yet...

Support for cross-application communication / shared data

At the very minimum, this should be permitted for apps released by the same developer

Major innovations to Springboard, since it is becoming unwieldy on iPhones with many apps installed

'spotlight' style launching would be good
It could be arranged in a grid pattern so you could flip left, right, up or down or diagonally. A faint arrow could show you where centre is. Pressing home would take you back to the centre screen.
For up to 9 screens you would be no more than 2 flips from any other screen.
A gesture (like pinch) could be used to zoom out the spring board, like spaces to see 4 page or further out for 6 and then 9 pages. Tap on a page to zoom back.
You should be able to name spaces, so you could see text on each space. e.g. 'Games', 'Financial', etc...

Utility class for loading a UIImage without caching

2009-01-04T18:15:00.000-07:00

I've checked in a utility class for UIImage, which adds a new category to load images without caching . You use it as follows:

UIImage *image = [UIImage imageFromResource:@"my_image.png"];

You should only do this if there is a specific reason you do not want caching.

New cocos2d release

2008-12-26T14:40:00.000-07:00

See this post and get the premier 2D iPhone graphics library here.

New repository for iPhone examples created

2008-11-30T16:00:00.000-07:00

I've created a google code project , which I will use to post working mini-examples for iPhone developers.

The first installment is an example of using parallel ARM assembler in XCode to do basic processing of an ARGB image. I'll post a follow up with more detail. In summary, for each iteration, the routine

reads 16 bytes into 4, 32-bit registers (r2-r5) in 1 ldmia (load multiple) instruction,
processes the 16 bytes in 4 uqadd8 instructions, which equates to 4 pixels (ARGB).
stores the 16 bytes back to memory, and increments the buffer point in 1 stmia instruction

C64 for iPhone: There and back again

2008-11-15T15:56:00.003-07:00

Lately, I've had a chance to play some more with the iPhone (and I'm really enjoying XCode). The title, 'There and back again' eludes to the fact that in my quest to have a C64 on my iPhone, I started by porting Frodo, and then ported VICE. Now I'm 'back again' to Frodo.

Why?

I'll spare all the technical details for another post, but essentially the design of VICE (written in C) was not optimal for achieving playable performance out of a mobile CPU, like the iPhone. Frodo is a much smaller code base, written in C++ and designed in such a way that GCC's optimizations were fully utilised. I've spent time fixing some lingering issues with Frodo the last few days and now have iFrodo running 100% emulation speed at 25fps. I could run the emulator at 50fps, but frame buffer access is not optimal in the public SDK today, adding significant overhead. When running Frodo in 'turbo' mode, the emulator runs up to 900%!

I fixed an issue in the 'fake' drive emulation, where many games would fail to load using the 'fast drive mode'. This mode typically loads games in less than a couple of seconds at 100% emulation (even fast in turbo mode). I found they would work when enabling the 'Emulate 1571' switch, but given this emulates the 6502 CPU similar to a real 1571 drive, it can take many minutes to load a game in some cases. With that fix, the majority of games can load in fast mode. For usability, I also updated the fast drive code to alert the user when a game requires the 1571 emulation, so it doesn't just crash, leaving you wondering.

I also found out that some games are cracked in such a way that are incompatible with Frodo (such as Wizball and Fist II). I located alternative images of these games that worked, so Frodo is a go!

I resolved some remaining issues with the 'touch stick', and now can play games effectively. I think the idea has turned out great in practice. I chose this over the more popular on-screen 'D' pad and fire button, which forces the user to press a specifically targeted area, that is difficult to master without any tactile feedback. What lead to my design was that I attended a User Experience seminar, presented by Chris Nodder of Nielsen Norman Group (ironically, their website appears to be an example of poor design) and learnt about Fitts' Law. I've alloted the whole screen to joystick function, meaning you can watch the game and not your fingers.

The majority of work left is related to

Saving state when the app is terminated (such as by the Home button or an incoming phone call).
save / load named game state and
landscape mode.

I'll create a video demonstration the emulator in action on a physical device very soon.

This all helps with some of the issues I had with mSID too - so I'll post an update on that shortly.

The GPL and iPhone development

2008-10-29T01:39:00.020-07:00

I have read through the GPLv3, and whilst I am not a lawyer, it is worded without too much legalese. Here is my take in FAQ style, addressing some points of contention that I have come across on a number of forums.

Doesn't the signing certificate imply 'Tivoization'?

To clarify what 'Tivoization' is, included is the following from the article A Quick Guide to GPLv3:

Tivoization: Some companies have created various different kinds of devices that run GPLed software, and then rigged the hardware so that they can change the software that's running, but you cannot. If a device can run arbitrary software, it's a general-purpose computer, and its owner should control what it does. When a device thwarts you from doing that, we call that tivoization.

Many users are confusing the purpose of the signing certificate as a potential 'Tivoization' violation. Your signing certificate has nothing to do with a specific application, nor is your application tied to a specific signing certificate (unless you were to somehow code it as such). It is simply a requirement imposed by Apple that all 'official' applications be signed and it's purpose is to identify who built the application and that the application has come from a trusted source. Remember, anyone who has access to the SDK and has paid their dues can sign an application and install it on the physical device.

GPL requires all the source be released, but doesn't the NDA prevents this?

Apple has revised the NDA, and now it applies to unreleased versions of the SDK. At the time of publication, that would imply anything built on version 2.2 of the SDK (which is currently in beta) is under NDA.

GPL software requires all installation scripts ('Corresponding Source') be released, so don't I have to release my signing certificate, which would be a violation of Apple's license?

You do not need to release your signing certificate. The source code or the compiled application is not tied to the original author's signing key, and as mentioned elsewhere in this FAQ there is no 'Tivoization' violation. An individual can simply uninstall the original version of GPL'd application, compile using XCode and reinstall their modified version. Releasing the compilable XCode project will ensure your project is in compliance with the GPL, as this handles the installation of the software to the device.

You must pay to enter the Apple Developer Program, is this a violation of the GPL?

Yes, and you had to pay to get an iPhone in the first place. You also had to pay for an Apple computer running OS X to do official iPhone development. Paying to enter the developer program is another cost, but this is not a violation with regards to GPL'd software. Once you've paid your dues and received your own signing certificate, you can compile and install the GPL'd software you downloaded.

Ultimately, the user still has complete freedom to modify and install the GPL'd application - and that is the essence of GPL'd software.

!NDA

2008-10-05T21:59:00.005-07:00

This is a huge boon for all the GPL software out there. The more I researched the issue, the more I realised I wasn't going to be able to release any GPL'd software via the App Store until it was lifted.

Also, I've been quiet lately - not because I've lost interest in the iPhone, but my day job has required more of my personal time lately - admittedly, the GPL issue didn't help. I have been talking to Andreas, who is the author of SIDPLAY for OS X. We are going collaborate, which should result in a great version of SIDPLAY for the iPhone. Unfortunately, we have both been very busy lately, so things have been moving slowly, but don't despair, as in the coming months I'm sure we will be able to finish off SIDPLAY for the iPhone.

mSID and the High Voltage SID Collection

2008-09-02T01:24:00.002-07:00

The default format for running the High Voltage SID Collection (HVSC) on the device will be in a ZIP archive. The HVSC is about 65mb zip'ed up, which in turn contains a 200mb uncompressed archive, C64Music.zip. This inner archive contains over 35,000 files. It is impractical to expect that will extract on the device in a reasonable amount of time, hence the decision to stick with an archive. I will provide a re-compressed version of C64Music.zip, which you will be able to download via mSID and from then on, one should be able to download the HVSC updates direct from their site. I will also include a sqlite database of all the files, in order to improve performance for searching. I expect that long-term, the role of the database will grow to become your store of ratings, play lists, etc. Given it's sqlite, it will be open to reading and updating.

From an implementation perspective, I have pulled together the necessary code to be able to read a ZIP archive as Cocoa does not have ZIP support built in. I have built this as a .framework, and will probably open-source it at some point.

The browser will use the familiar hierarchical navigation built in to the iPhone.

Introducing mSID

2008-09-01T00:56:00.016-07:00

'm' what? mobile-SID!

Given the setback in rendering iC64 usable, I wanted to tackle something that would generate results relatively quickly. There has been a modest amount of interest in a SID player for the iPhone (it is a mobile music player after all) and so mSID has begun.

As of this evening, I have played Wizball on the the device, and to my delight it works like a charm. There is no user interface to speak of yet so no screenshots; however, I can focus on the design now that playback is not going to be an issue.

I'll be working on brining mSID to the App Store in the coming weeks.

How will I get songs on the device?

I have several games that allow downloading of content (new levels) from the internet, so I will provide the same feature.

What about STIL and song length support?

Of course.

What's going on with iC64?

2008-09-01T00:20:00.007-07:00

Lot's of emails and posts asking how the emulator, iC64 is coming along, so I wanted to post a short update. Firstly, iC64 is still alive, but took a back seat for a while due to life commitments, both work (travel and major release going out the door) and much needed vacation. I am beginning to find some spare time, allowing me to return to my hobby.

What's the problem?

I'm running into some significant performance problems running iC64 on the device. Let's not forget this is a mobile device, and VICE is designed to run on some pretty hefty hardware. I believe the iPhone can do it, but it's going to take some work on my part to get it there.

What can you do about it?

I've been staring at ARM assembly (not Thumb) and monitored the device with Shark, which is a tool used for performance analysis. It's obvious that there simply aren't enough cycles to run the emulator as it stands today.

Part of the issue is that VICE is broken into 100's of small files for modularity, but that causes problems accessing shared static data across object files. What I'm finding is that a variable 'foo' declared in file 'a' and accessed in file 'b' generates three memory accesses. Two indirect reads before getting the value. Ouch. I've found ways to mitigate this, but it's going to require a fair amount of refactoring. I'll start with the hot-spots first and work my way on from there, until I've squeezed out the performance I need. Unfortunately, how long this will take is an unknown, so I'll post updates as I progress.

Stay tuned.

I'm in!

2008-07-12T14:22:00.002-07:00

It seems with the release of iPhone 2.0, Apple opened the flood gates to the iPhone developer programme on Friday. I received my acceptance email late in the day. I've subsequently paid my dues ($99 USD) and now have iC64 running on the device. This brief experience leads to a separate post about compiler settings for hi-performance applications on the iPhone.

iFrodo is dead, long live iC64

2008-06-20T10:02:00.005-07:00

The Obituary

To whom it may concern:

After repeated gaming accidents, it is with sincere regret that iFrodo is no longer. A team of 1 and countless hours spent operating in the debugger simply could not save it.

And for all that you, iFrodo have taught me about the iPhone SDK, I am forever in your debt.

The Diagnosis

During routine exercises (testing), iFrodo was consistently passing out. The first occurrences were running Wizball and Fist II, which caused iFrodo to report an invalid opcode. In an attempt to eliminate any genetic issues, I decided to check family history, which included WinFrodo and their .NET cousin, sharp-64; however, they showed almost identical symptoms.

Next, to eliminate the possibility the D64 images were corrupt, I tested the offending games using iFrodo's distant relatives, CCS64 (PC), WinVICE (PC), VICE (OSX) and Power64 (OSX) all of which had no issues.

I spent many hours and explored many avenues, but could not come up with the elusive fix.

The Future

I have completed a port of VICE to the iPhone, iC64. Any suggestions for an alternative name for the 'product' are welcome. More importantly, all functionality that was present in iFrodo is available in iC64, with the exception of the previously mentioned bugs.

A picture is worth a thousand words and a video is worth a lot more:

Update: Video now plays the emulator sound. Forgive the voice-over, as it is a little quite when the sound is on.

iFrodo - Keyboard mania

2008-06-04T01:00:00.002-07:00

I've now successfully hooked the keyboard up to iFrodo. Without a doubt, the experience is much better.

I've uploaded a short video to youtube to demonstrate the new functionality in action. Forgive the green text over the video implying 'please register me, you bad, bad individual' - I will buy a copy of the software and redo the video.

iFrodo Keyboard: Working

2008-06-03T01:10:00.001-07:00

I've now completed the keyboard to the point where I can load and run programs via the emulator, including support for multiple layouts.

I will add the Fn keys next and then hook it all up to the emulator, removing the clunky NSTextField and built-in keyboard. I short video will follow with this milestone.

A longer term usability goal is to show the actual 'drawing' keys when the shift or C= key is pressed, but gathering all the images of the keys is a lot of work for a rainy day.

The default layout:

The number and glyph layout (Fn keys will go here):

iFrodo - More keyboard features

2008-06-01T02:52:00.001-07:00

A lot of 'under the hood' changes have been made to the keyboard, including support for left, centre and right snap regions (note the shift and del keys), sticky keys (like the shift or the Commodore key) and multiple layouts. Multiple layouts will lead to support for switching between the QWERTY or number layout.

I will be pretty close to hooking the basic keyboard up to the Frodo emulator.

iFrodo - The keyboard is shaping up

2008-05-28T00:03:00.003-07:00

I have the basic framework for rendering any virtual keyboard (in a resolution independent manner) almost completed. The keyboard supports touch events; however, I now have to complete the shift-key support and a few other enhancements for switching layouts (to numbers, etc).

For nostalgia, the colors are pulled from numerous pictures of Commodore 64 keyboards:

Update: I will be including full support for all the Commodore 64 keys (C64 key, del, clr, etc)

iControlPad

2008-05-25T02:36:00.006-07:00

I'll definitely purchase one of these when they are released. In addition, I'll be particularly interested in adding support for the device to iFrodo, using their provided SDK. assuming I can access the unit via the Apple SDK

iFrodo - 'Touch-Stick' demonstration

2008-05-22T22:51:00.010-07:00

Attached is a short video to demonstrate the joystick (touch-stick) emulation in action using the iPhone Simulator and Impossible Mission by Epyx. Hopefully, once I receive my developer certificate, the experience will be functional on the device. I do plan to support landscape mode, along with switching the position of the fire button area.

SQLite management tool

2008-05-17T20:21:00.001-07:00

For those using SQLite (particularly relevant given it's the default database engine for the iPhone / iPod Touch), here is a great tool for visually managing you database. What is most impressive is the tool is a Firefox add-on, compatible with versions 2.0 - 3.0.*.

iFrodo - Let there be sound

2008-05-12T23:51:00.003-07:00

Watch and listen.

Be aware that the video and audio are greatly compressed on it's journey to YouTube. It sounds significantly better on the simulator.

iFrodo - progress

2008-05-11T22:05:00.003-07:00

Where are we today?

Graphics: Done
- I am using the scan-line mode for the emulator, but I may change to the single-cycle mode after some performance testing. So far it screams on the simulator. I could deploy both versions of the emulator, but that would require considerable work. Hopefully this runs with similar performance characteristics as the real device. I won't know until Apple accepts my application...

User Experience : None
- Portrait: Partial
  - In this mode, the full keyboard will be available, as well as playing games
- Landscape: None
  - This mode will be primarily for playing games, with the availability of the joystick and probably the Fn keys. Switching to portrait will be necessary to perform more work with the keyboard.

Audio : Researching
- I've found that the Audio Queue Services look to be exactly what I need. Frodo was conveniently built to support sound 'drivers', so shouldn't be too much work here.

Input, Keyboard : Planning
- What I have to day is a 'line editor', using the built-in keyboard (as you can see in the video clip). I type a line of text and 'send' it to the emulator. This is just a debugging tool and will go away before the release.
- The iPhone SDK and associated frameworks do not allow the built-in keyboard to be repurposed, so I'm going to have to roll my own. This is probably okay, as I'd like to make it generic enough to work with other emulators in the future - think Atari 800XL and Apple II.

Input, Joystick : Planning
- Two options
  - Multi-Touch: I have some ideas for joystick support, which I'll document in a future post - it's going to take advantage of the multi-touch interface and the entire screen surface - 'Touch Stick'.
  - Accelerometer: Something similar to what was shown for the initial SDK release for the space shoot-em-up style game.

Show and Tell
Here is a short video showing 'Hello World' and running IK+ (International Karate +). As mentioned above, it uses the built in keyboard, in 'line mode', which is simply a debugging tool, until I build the UI. No sound either, but that is underway. I'll follow up with another video once sound is activated.

Note about the video recording: It was recorded at 15 frames/sec to limit the size up to youtube; however, the emulator runs at 30 fps without breaking a sweat.

iFrodo - Success!

2008-05-08T00:46:00.003-07:00

After getting over the Objective C and Cocoa learning hump, I have a running C64 on the iPhone simulator.
Next will be to create a UI to manage the user experience, such as

Save / Resume state
File browser for disk / tape images
- Ability to 'auto launch' games
Virtual keyboard and joystick

iFrodo - porting C64 emulator to the iPhone / iPod Touch

2008-05-05T10:37:00.005-07:00

As an Objective-C / Objective-C++ / XCode / iPhone development learning experience, I have began porting the Frodo C64 emulator to the mobile OS X platform. I chose Frodo, as I have experience with this code-base, and there is a certain satisfaction of seeing the READY prompt for the first time.

What will some of the challenges be?
Display

The iPhone screen is 320x480 pixels. The C64 had a 320x200 pixel display, so ideally the top of the screen will represent the C64's virtual display, without the need to scale.

Input

The C64 had a keyboard and games were typically played with a joystick. We have 280 pixels left to represent these devices. I'm not sure if I can display the iPhone's virtual keyboard, and it is unlikely I can reconfigure the keys. Therefore, I'm probably going to have to design my own virtual keyboard image.
For the joystick, I'll probably model something off the NES emulator, with a virtual D-Pad and fire button.

Where are we today?
I have reached my first milestone, which was to create an XCode project and compile the basic Frodo engine for the iPhone platform.

What is next?
An emulator typically virtualises numerous devices of the original hardware, so the first I will tackle is the display, as this provides the most immediate rewards. We need an in-memory buffer to write the output of the virtual display device, which finally is rendered to host screen. I experimented with the OpenGL ES API, but found that Core Graphics / CGImage will do the job just fine.

Last night I achieved my goal of writing to an in-memory buffer and rendering this to the iPhone screen. Next is to incorporate this test code into the Frodo engine, to get a C64 running. Screenshots will follow.

SDL coming to iPhone / Touch

2008-05-04T01:36:00.003-07:00

I've learnt that Google SoC is sponsoring a project to bring SDL to the iPhone / iPod Touch. Great news!

Can't see drivers\etc folder or hosts file in Vista x64?

2008-04-29T12:21:00.004-07:00

I've just joined the Windows 64-bit OS club and ran into a frustrating issue, whereby I couldn't edit my hosts file (%systemroot%\system32\drivers\etc) using Altap Salamander. If I used Vista's Explorer, there it was.

Now, what's important to understand is Salamander is a 32-bit Windows application and the Wow64 layer is redirecting 32-bit applications to %systemroot%\SysWOW64. I remembered I 'knew' this, but surely there must be a way for a 32-bit application to see the native system32 folder, for file management operations.

Turns out there is. Create a folder %systemroot%\sysnative, and navigate via this to see the native system32 folder from within 32-bit applications.

Vista x64 officially supported on Apple hardware

2008-04-26T10:54:00.003-07:00

The Vista x64 update for Boot Camp is available here. Perfect timing, as I'll be getting a new 17" Macbook Pro for 'research' at work in the coming weeks, and was hoping to dual-boot with Vista x64.

SIDPLAY 4.0 for OS X 10.5

2008-04-26T09:55:00.006-07:00

For all those old Commodore 64 chip-tune fans out there, take a look at the re-released SIDPLAY for OS X.

In Summary:

Rewrite of the UI
iTunes look and feel
Export to mp3, AAC, AIFF and Apple Lossless
Integrated Spotlight search
Core Audio support

Get latest code from TFS via command line

2008-04-25T12:36:00.003-07:00

I tend to work from the command line for managing builds and source control. For my day job, we use TFS, so I use the Team Foundation command-line utility tf.exe with the 'get' option.

Be sure to start a Visual Studio 200x command-line prompt, to ensure tf.exe is available in the search path.

tf get will recursively grab changes from the current directory, so you can be somewhat selective.

If there are any conflicts, the familiar conflict dialog is presented, allowing you to determine the appropriate resolution.

Improve MSBuild performance with multi-processor builds

2008-04-25T12:28:00.004-07:00

Using the command line :\>msbuild /m[:max cores] will automatically take advantage of multiple cores if they are available. Without the max cores parameter, msbuild will use all available cores. I've seen some recent posts on this topic, and wanted to point out I have been using this feature for a number of months, and can confirm noticable performance gains on my dual-core laptop. A big issue for laptops is the IO performance, so if you have slower drives, you may not see the same gains. Have not had a chance to try it on my 8-core Mac Pro :-)

Recent Mono / MS JIT observations with #64

2007-07-16T12:58:00.001-07:00

Some time ago, I had run #64 using the Mono and MS runtimes. Judging on the timeframe of my post, I expect I was running version 1.2.3.

Everything worked great; however, there was a fairly significant delta in performance between the two, Mono falling quite short of the MS runtime. It didn't concern me at all, as the emulator was still running at 100% the performance of the original C64.

I am now running version 1.2.4.

I ran the emulator under both runtimes for about 90 seconds, and monitored the CPU usage under the Performance Monitor. I started monitoring after the emulator had fully started up, to exclude JIT overhead.

Command Line		CPU Usage (%)
	Average	Minimum	Maximum
mono TestHost.exe	38.813	29.687	67.187
TestHost.exe	45.229	21.875	59.375

Granted, these tests aren't entirely scientific, but for 90 seconds, the average is quite different, in Mono's favour.

Could the recent patches to the MS runtime be causing the drop in performance?

UPDATE:

Turns out that my machine was set 'Maximum Battery' mode. In 'Maximum Performance' mode. I wonder why Mono performs better in this scenario?

When reverted to 'Maximum Performance', the CPU usage of Mono didn't change (~38%) and is fairly consistent:

whereas the MS JIT jumps from 20% to 40% in cycles, as follows.

Each vertical line represents 4 seconds.

Google Bookmarks 'bookmarklet' link in Safari

2007-07-11T10:34:00.001-07:00

If you are unable to use the Google Bookmarks 'bookmarklet' in Safari, the workaround is to manually add a bookmark and use the following javascript code for the address.

javascript:(function(){a=window;b=document;c=encodeURIComponent;d=a.open("http://www.google.com/bookmarks/mark?op=edit&output=popup&bkmk="+c(b.location)+"&title="+c(b.title),"bkmk_popup","left="+((a.screenX||a.screenLeft)+10)+",top="+((a.screenY||a.screenTop)+10)+",height=420px,width=550px,resizable=1,alwaysRaised=1");a.setTimeout(function(){d.focus()},300)})();

The issue is the spaces and quotes in the original javascript code do not appear to be correctly decoded in Safari.

Mono 1.2.4 released

2007-05-15T13:48:00.001-07:00

According to the Mono Project feed, Mono 1.2.4 has been released. Check out the link as there are lots of improvements.

GraphicsDeviceCapabilities.CompareCaps

2007-05-15T10:53:00.001-07:00

Implemented CompareCaps a few days ago, which are exposed via the AlphaCompareCapabilities and DepthBufferCompareCapabilities. These indicate the comparison operators supported for the alpha and depth buffers. Their OpenGL equivalents are glAlphaFunc and glDepthFunc respectively.

The implementation was easy, since they were all supported since v1.0 of OpenGL and therefore just return true.

As per MSDN documentation,

Represents comparison capabilities.

Property	OpenGL Extension or Minimum Version
SupportsAlways	v1.0
SupportsEqual	v1.0
SupportsGreater	v1.0
SupportsGreaterEqual	v1.0
SupportsLess	v1.0
SupportsLessEqual	v1.0
SupportsNever	v1.0
SupportsNotEqual	v1.0

MonoXna - what version of OpenGL?

2007-05-15T10:12:00.001-07:00

A few posts on the MonoXna groups generated an interesting discussion related to implementing the GraphicsDeviceCapabilities API. What is the minimum version of OpenGL we should (or even can) support?

There will be a number of features expected of the base Xna implementation, which will not be available in earlier versions of OpenGL.

So far, from what I can tell we will need

v1.5 with a significant set of ARB/EXT extensions
2.0 with the pixel buffer, and a few other ARB/EXT extensions
2.1+

Should we go straight for 2.0 or try to target 1.5?

Altap Salamander 2.5 has been released

2007-05-14T10:17:00.001-07:00

I am a long time user of Orthodox File Managers, which can generally be categorised as a two pane view of a hierarchical storage system. Examples of these include file, registry, archive (zip, rar, etc) ftp, nfs, etc. I used Far for some time, which has a fantastic plug-in model, but was not progressing. I changed to Salamander 2.5 Beta well over a year ago, and have never looked back. Now that it has been released, you can pick it up for $30 USD.

Whilst Far is somewhat nostalgic of DOS file managers, like Norton and XTree, we live in a world of WYSIWYG, and I find character mode doesn't cut it when previewing files. This is where the move to Salamander was welcomed. It's still missing features a loved in Far, but if Altap ever release their SDK, I'll be the first to jump in.

For OSX I have been experimenting with Disk Order 2.5 - it still has a ways to go, lacking many of the standard key strokes one expects from a 2 pane file manager, but is by far the best option on the Apple platform today.

GraphicsDeviceCapabilities.BlendCaps structure

2007-05-09T00:47:00.001-07:00

As per MSDN documentation,

Represents the supported blend capabilities.

There are 14 capabilities for this structure, as follows

Property	OpenGL Extension or Minimum Version
SupportsBlendFactor	V1.4 \|\| ARB_imaging
SupportsBothInverseSourceAlpha	V1.0
SupportsBothSourceAlpha	V1.0
SupportsDestinationAlpha	V1.0
SupportsDestinationColor	V1.4
SupportsInverseDestinationAlpha	V1.0
SupportsInverseDestinationColor	V1.4
SupportsInverseSourceAlpha	V1.0
SupportsInverseSourceColor	V1.4
SupportsOne	V1.0
SupportsSourceAlpha	V1.0
SupportsSourceAlphaSat	V1.0
SupportsSourceColor	V1.4
SupportsZero	V1.0

If these are not correct or you can help fill in the blanks, comments are welcome.

Tao 2.0 and SDL.NET 6.0 released

2007-05-07T11:33:00.001-07:00

Great to see that Tao 2.0 and SDL.NET 6.0 were released over the weekend.

The pre-release versions are currently in use by MonoXna, so I look forward to updating to the released versions as soon as possible.

GraphicsDeviceCapabilities.AddressCaps structure

2007-05-04T09:05:00.001-07:00

As per MSDN documentation,

Represents the texture addressing capabilities for Texture structures

There are 6 capabilities for this structure, as follows

Property	OpenGL Extension or Minimum Version	Notes
SupportsBorder	GL_ARB_texture_border_clamp	Gets a value indicating whether the device supports the setting of coordinates outside the range [0.0, 1.0] to the border color.
SupportsClamp	GL_EXT_texture_edge_clamp	Gets a value indicating whether the device supports the clamping of textures to addresses.
SupportsIndependentUV	Unknown	Gets a value indicating whether the device can separate the texture-addressing modes of the texture's u and v coordinates.
SupportsMirror	GL_EXT_texture_mirror_clamp	Gets a value indicating whether a device can mirror textures to addresses.
SupportsMirrorOnce	GL_EXT_texture_mirror_clamp + GL_ATI_texture_mirror_once	Gets a value indicating whether a device can take the absolute value of the texture coordinate (thus, mirroring around 0) and then clamp to the maximum value.
SupportsWrap	Unknown	Gets a value indicating whether a device can wrap textures to addresses.

If these are not correct or you can help fill in the blanks, comments are welcome.

GraphicsDeviceCapabilities and OpenGL

2007-05-04T08:06:00.001-07:00

Part of the process of implementing GraphicsDevice and GraphicsAdapter involve a number of related classes. The first I'm going to tackle is GraphicsDeviceCapabilities.

As per the MSDN documentation, this class represents the capabilities for the hardware. It is made up of no less than 18 structures, defined with the GraphicsDeviceCapabilities class definition. I expect I may run in to a few cross platform issues whilst implementing some of the queries, so it will be an interesting exercise.

Over the coming days / weeks / whenever, I'm going to document each structure as a separate article and on the Mono.XNA groups as to how I've obtained the capabilities, with references to articles, etc - hopefully garnering feedback from the Mono.XNA team and other OpenGL / DirectX experts.

I've also created a page on the mono.xna discussion group, to track and document our implementation of the various GraphicsDeviceCapabilities structures.

C# partial classes and file naming conventions

2007-05-03T00:59:00.001-07:00

I find that partial classes in C# have more uses than just the automatic .Designer.cs created by numerous designers in your IDE of choice.

In certain scenarios, I will create partial classes and use multiple files. I consider the [Class Name].cs as the primary file and [Class Name].[Name].cs as the satellite files. The name is up to you, but should succinctly indicate the function and preferrably is one word.

I've found in practice it has worked very well, with regards to maintenance and reducing the time to understand a class. I've been using this convention for a while, and felt it was time to review, given a few ideas I've been tossing around.

Currently I only use it for classes that are large enough to warrant separating out functional areas. Additionally, if the class implements a complex interface or a related set of interfaces I may move these to a separate file.

I consider this is appropriate for larger classes and therefore you would always start off with a single file and as your implementation grows, reconsider if it is worth separating.

Limiting the public interface of the class to the primary file, so it is very easy for a developer to view the file and determine the public API
- Move protected and private members to a separate file
  - [Class Name].private.cs
- When a class has a significant number of private and protected members, separate as
  - [Class Name].private.cs
  - [Class Name].protected.cs

I know regions can provide similar functionality, assuming your editor supports outlining, but I tend to see them either overused or not at all. When viewing a file in [insert simple text viewer of your choice], regions provide little value.

I usually limit regions to:

#region private fields #region constructors #region public members #region protected members #region private members

Thoughts?

GamePad and Color implementations for Mono XNA

2007-05-02T00:54:00.001-07:00

Just posted some updates to the repository tonight, now that I have write access. This is really going to help me, in moving things along whilst I have a little time to contribute right now.

GamePad
- As previously mentioned, GamePad has a preliminary implementation of the two dead zone algorithms now
- Unit tests (where possible) have also been submitted
Color
- The initial source appeard to come from the System.Drawing namespace in the mono implementation of the BCL, so it was a little heavy for XNA. I've submitted a revised versions that is greatly simplified and is now 100% complete, with associated unit tests.
- These unit tests helped me figure out a few things about the MS implementation, including the fact that Vector3 / Vector4 conversions use rounding rather than truncation.

I've started work on GraphicsDevice and Graphics adapter, but these will require acquiring a more thorough knowledge of OpenGL. To assist with this process, I've ordered the Red Book and OpenGL Distilled.

Update: I've added the OpenGL: Reference Manual to that order

I'll post some discovery in the coming days.

GamePad Dead Zone functionality

2007-04-30T11:24:00.001-07:00

The last couple of nights I've been working on the dead zone functionality for the GamePad class in MonoXna, which was an interesting excursion off into formula's and equations I haven't been exposed to for many years.

I read an post by Shawn Hargreaves, which gave me a good start on understanding the MSXNA implementation. I've now implemented the basic functionality of the two algorithms, IndependentAxis and Circular.

I used Shawn's formula for the IndependentAxis deadzone calculation and here is a plot of this:

Independent Axis - Plot X	Independent Axis - Plot XY

I ended up using a variation of Shawn's formula for the IndependentAxis and using a bit of trig, ended up with the following for the Circular implementation:

/// <summary>
/// As per http://blogs.msdn.com/shawnhar/archive/2007/03/28/gamepads-suck.aspx
/// Both <paramref name="x"/> and <paramref name="y"/> parameters assumed to be between -1.0 and 1.0
/// </summary>
static Vector2 CircularAxisDeadZone(float x, float y)
{
    float dist = (float)Math.Max(Math.Sqrt(x * x + y * y), EPSILON);
    float deadZone = Math.Max(dist - DEADZONE, 0)/(MAX_DIST - DEADZONE)/dist;
    return new Vector2(x*deadZone, y*deadZone);
}

The MSDN documentation states that the Circular algorithm uses the combined position of both joystick axes to make the deadzone calculation. So,

We calculate the distance from the centre, and if zero, returning EPSILON, so we never divide by zero.
If we're in the DEADZONE (=0.2f), we'll clamp to 0, otherwise we'll create a weighting factor for the deadzone calculation
return a Vector2 with the new x,y values

Here is a plot of our IndependentAxis calculation:

Circular - Plot XY

#64 - Commodore 64 emulator on Mono

2007-04-23T09:55:00.001-07:00

With the recent success of running Pong on Mono.Xna, on the mono runtime, I decided to try my port of #64. I was pleasantly surprised to see it run without recompilation or any issues on the mono runtime.

My first observation was the not insignificant increase in CPU usage over running on the Microsoft JIT.

I haven't had a chance to dig into things yet, and it may not be for a while, some areas I will look at are:

switch statements
- Like many emulators, #64 has a number of large switch statements to emulate the various CPUs (6502, 6510, 6581, 6569, etc), and these are perhaps not as well optimized as the MS JIT.
inlining
- Whilst I inlined some code, the other possibility is that the mono JIT may not be as aggressive when inlining code.
floating point
- The sound emulation does use a significant amount of floating point code.
memory allocation
- I am pretty sure the memory allocation is minimal, and suspect the mono memory allocator is just a pointer being incremented, much like the MS implementation

It will be an interesting journey into the inners of the code generation and optimizations, since I'm not familiar with all the optimization techniques. I'll blog about my experiences when the time comes.

Running Pong under Mono.Xna on mono

2007-04-19T00:05:00.001-07:00

It is official; we've successfully run Pong under Mono.Xna! It is running with either the Microsoft or mono runtimes!

Last night, I completed most of the core functionality within the Microsoft.Xna.Framework.Game.dll assembly, and wrote a number of unit tests to exercise the Game logic.

Looks and runs identical to it's counterpart running under the Microsoft implementation of Xna.

~~My next step is to run it under the mono runtime on my windows machine and then follow with running it on my MacBook Pro.~~ This is completed. Now for my Macbook Pro.

I will submit patches after I complete the additional classes I stubbed to get this basically working.

Mono.Xna - The beginning

2007-04-17T14:10:00.001-07:00

Why?

Given my interest in game programming, graphics and generally making computers do interesting things other than running enterprise software, I've starting contributing to the mono.xna team.

These series of posts, tagged with XNA, will chronicle my findings, exploration and implementation of my areas of the XNA framework.

Hardware and Development

I own a Macbook Pro and a Windows laptop, so I can develop and test on both these platforms. My primary development environment will be VS 2005, since I cannot live without Resharper 2.5, and think any serious software engineer should at least look at this tool.

And it begins...

After I obtained the current code, I found the best place to start was implementing the classes in Microsoft.Xna.Framework.Game.dll, since this assembly was largely unimplemented, and is really one of the core entry points for a consumer of this framework.

Based on previous conversations in the discussion group, Tao and SDL.NET are the cross-platform APIs of choice. This was great to see, given I've previously used and contributed to SDL.NET for my C# implementation of the Frodo C64 emulator, and found it easy to use.

Command Prompt : Set the title for easier ALT-TAB

2007-04-04T13:47:00.000-07:00

If you are like me, and run several command prompts at any given time, rooted at different folders, it turns into a game of chance, when 'ALT-TAB'ing. "Visual Studio 2005 Command Prompt" is a little too generic.

I'm in the habit of using the title command to use something more meaningful:

Generic copy object using Lightweight Code Generation

2007-03-25T11:42:00.000-07:00

I was involved in a discussion on microsoft.public.dotnet.framework.clr regarding reflection and lightweight code generation. The original topic was in regards to a 2x slowdown using reflection in .NET 2.0 over 1.1, which was ultimately recognised as a bug by the Microsoft CLR team. I brought up the possibility of using LCG as an alternative in 2.0, and the one of the contributers presented some good examples of using Delegate.CreateDelegate. Here I wanted to present a use of LCG to copy all the public read/write properties of any object. The meat of the example was the GenerateCopyDelegate function, as follows:

static CopyPublicPropertiesDelegate<T> GenerateCopyDelegate<T>()

    Type type = typeof(T);

    DynamicMethod dm = new DynamicMethod(type.Name, null, new Type[] { typeof(T), typeof(T) }, typeof(Program).Module);

    ILGenerator il = dm.GetILGenerator();

    PropertyInfo[] props = type.GetProperties(BindingFlags.Public | BindingFlags.Instance);

    foreach (PropertyInfo prop in props)

        if (prop.CanRead && prop.CanWrite)

            il.Emit(OpCodes.Ldarg_1);

            il.Emit(OpCodes.Ldarg_0);

            il.EmitCall(OpCodes.Callvirt, prop.GetGetMethod(), null);

            il.EmitCall(OpCodes.Callvirt, prop.GetSetMethod(), null);

    il.Emit(OpCodes.Ret);

    return (CopyPublicPropertiesDelegate<T>)dm.CreateDelegate(typeof(CopyPublicPropertiesDelegate<T>));

Method	Calls (per iteration)			Total Calls
	Delegate	get_*	set_*
LCG	1	6	6	13,000,000
Delegate	12	6	6	24,000,000

A simple console logger

2007-03-19T13:49:00.000-07:00

A very simple class for logging output to the console. What's special about it?

Intended for the console only.

I wanted to keep the code very simple.

Uses colour to differentiate the log message based on type: INFO, WARN or ERROR.
Supports immediate or buffered output

All colour output is honoured in buffered mode.
Use Logger.Flush() to display pending output.

Get it here.

A better FindControl: FindControlOf

2007-03-19T01:41:00.000-07:00

Whilst helping a colleague with some ASP.NET questions, I found a number of blocks enumerating the Controls collection for specific control types. Having to enumerate and determine the control type can become quite tedious, so I provided an iterator to simplify:

   1:  static IEnumerable FindControlsOf<T>(Control parent)

   2:      where T : class

   3:  {

   4:      foreach (Control control in parent.Controls)

   5:      {

   6:          T item = control as T;

   7:          if (item != null)

   8:          {

   9:              yield return item;

  10:          }

11:

  12:          // Don't enumerate the children, if control is type T as an optimization, since our

  13:          // typical search criteria is not container controls.  If the criteria changes,

  14:          // remove the "item == null" check.

  15:          if (item == null && control.Controls.Count > 0)

  16:          {

  17:              foreach (T child in FindControlsOf<T>(control))

  18:              {

  19:                  yield return child;

  20:              }

  21:          }

  22:      }

  23:  }

Making it easy to enumerate controls of a specific type:

   1:  foreach (MyControl sa in FindControlsOf<MyControl>(pnlMyContainer))

   2:  {

   3:      ...

   4:  }

I should point out that this code generates recursive state machines depending on the depth of the hierarchy of controls, so it should be used carefully.

Caching and O/R mapping

2006-08-31T22:48:00.000-07:00

I was forwarded an article titled Why a cache in an O/R mapper doesn't make it fetch data faster and felt the need redirect the audience to some examples demonstrating the benefits of a well designed caching systems.

The author appears to be concerned you don't know if the data in the cache is dirty. Although, fails to point at the reason for this uncertainty. There are many reasons this could be the case, but why not provide an API or services for external applications, forcing them through you O/R framework (perhaps in an abstracted way), ensuring the cache is kept up to date. I've read several articles (and my own tests with applications I've worked with) that demonstrate typical applications perform a greater percentage of reads than writes, further suggesting a cache is beneficial.

What’s a cache?

I’d like to refer to this definition, in particular “A cache stores recently-used information in a place where it can be accessed extremely fast.”. I disagree that the Identity Map (or uniqing) Frans refers to is necessarily the only cache used in O/R frameworks.

Caches and queries: more overhead than efficiency

Quoting from the author’s article,

So, when does this efficiency the myth talks about occur exactly? Well, almost never. In fact, using a cache is often less efficient. I said almost, as there are situations where a cache can help, though these are minor or require a lot of consessions.

NHibernate (Hibernate ported to .NET) was developed with 2^nd level caching at its core, allowing you to control the caching options per class, such as whether it writes-through or invalidates it's entry on a write to the database, which I think is important in demonstrating a cache’s benefits. Additionally, NHibernate can cache query results as a list of IDs, furthering the benefit of its cache and increasing the hits.

The 2^nd level cache is completely pluggable, so you have many options, including the ASP.NET cache. For even greater scalability, consider a distributed caching solution such as memcached or NCache. This is counter to the point ‘.NET’ doesn’t have support for cross process or server object awareness.True, it’s not built in to the framework, but solutions certainly exist.

I now want to direct you to this article, Hibernate: Truly Understanding the Second-Level and Query Caches, which goes in to all the detail you need to understand just how powerful a well designed O/R framework and caching subsystem can be.

In the author’s example, we could create a named NHibernate query, with appropriate parameters, represented in HQL as

IQuery query = session.CreateQuery("from Customer C inner join C.Orders O where C.Orders.size > 5 and O.ModifyDate > :date").SetDateTime("date", DateTime.Now.AddDays(-30));

For the cache to be effective and accurate, all changes to the database must notify the cache, so we know the entry in the cache is current, and therefore do not need to go to the database.

August CTP Sandcastle buildchm.bat update

2006-08-31T17:27:00.000-07:00

Here is an updated buildchm.bat that works with the August CTP of Microsoft Sandcastle. The steps were derived from the Sandcastle blog. If you need to add additional assembly dependencies, edit the .BAT file, and locate the large comment. Cheers

sharp-64 - A Commodore 64 emulator in C#

2006-08-30T21:40:00.000-07:00

What is it?

As of today, I've scoured the internet, and think it is fair to say sharp-64 (#64) is the first Commodore 64 emulator for the .NET platform. I could go as far to say it's the first 'emulator' for the CLR, given various searches on google turned up nothing on the subject of emulation related to .NET / C# / CLR. #64 is written entirely in C#, using SDL.NET as the sound and audio library. By using SDL.NET, I expect to build #64 as a cross platform executable.

Why?

I've always had an interest in emulation and software emulators for old 8-bit platforms, such as the Commodore 64 and Atari 800XL (both of which I had as a youngster). I hacked around both machines quite a bit as a kid, learning 6502 assembler and playing with their excellent graphics and sound capabilities (of their time).
I also was interested in seeing how well the CLR handled the demands of an emulator, and how much I could optimize the code to gain every ounce of performance out of it.

How?

I cannot claim to have written the emulator from scratch - the credit goes to the excellent Frodo project. A very well designed and modular emulator written in C++. It is also a cycle exact emulator, meaning all the 'emulated' chips are in sync with each other, to the clock cycle. This allows emulation of practically every hardware hack on a real C64. Given the nature of cycle-exact emulators, they also require a greater demand of the platform it is executed upon, allowing me to further push the CLR.

Where is it at today?

It is a fully functioning C64 emulator, with sound and 1541 emulation via D64 virtual disks and mapping a directory. Sound and graphics use SDL.NET, with an extension to SDL.NET audio support for access to the streaming audio capabilities of SDL.
It has no user interface, besides the C64 itself, and therfore is not user friendly, nor is it recommended for a wide audience. Yet.
I have found a couple games do not work; however, they do fail with the official Frodo emulator, suggesting bugs in the original emulation code. I will try to troubleshoot this when I get some time.
#64 does have a few unsafe blocks, using pointers to access the emulated RAM, however I am planning to create some conditional defines and make these optional, to see how much overhead is incurred by the array bounds checks.

How does it perform?

When speed limiting is turned off, sharp-64 runs at greater than 400% of an original C64 on my 1.83GHz Core Duo Dell notebook.
I have tried various demos, such as those with FLI, FLD, DYCP, open borders, multiplexed sprites, timing dependent decoders, fast loaders etc, and they work great.
This is currently a straight port, so we may even be able to get closer to the C++ native version, which runs about twice the speed after some optimizations.

Can I play?

The full source is currently available via svn on sourceforge.net, https://svn.sourceforge.net/svnroot/sharp-c64. There is no documentation on getting it running, so it's up to you to hack around for now. The place to modify which d64 or directory is used for the 1541 disk drive is in prefs.cs.

Screen shots?

CBM Basic

Decathalon

Blue Max