Beginning iPhone Development

This book truly falls into category (A) and explains step by step how to start developing for the iPhone. Dave Mark and Jeff LaMarche not only discuss the concepts of the iPhone SDK. They describe essential little details of the tools you will face during your work such as the key strokes you have to know when using the Interface Builder. Some examples span across several chapters but the authors encourage you to branch out and try things on your own.
It’s still for the iOS 3.0 and doesn’t cover multi tasking nor the retina display or universal apps for iPhone and iPad. But once you have mastered this book, looking up those features on the Dev Center should be easy.

Link to Amazon, Book’s Website

Cocoa Design Patterns

This book definitely is not for beginners, it belongs to category (B). Readers should have worked with Objective-C and other programming languages before to value this great book by Erik Buck and Donald Yacktman. Newcomers to the iPhone development can use this book to gain a deeper understanding of the APIs they will face. Some of the patterns of this book are rather common in other programming languages, others are more or less unique to Objective-C. Over time, you will adapt these patterns to improve the design of your own programs, too.

Link to Amazon, Book’s Website

Tapworthy

This book is a must-read for mobile app developers and you should definitely read it before you seriously think about the development of an iPhone app. Josh Clark shares his deep understanding of the mobile user. To make it clear: This is not a coding book! You will learn a lot about designing interfaces (C), guiding your users through your app and keep them using your app by fleshing out its core features. The colorfully illustrated book rounds out with interesting experience reports by the developers of the Facebook app, Twitterrific, Gowalla, Things, and the like.

Link to Amazon

Yes, there are a lot more exciting books covering iPhone development but these are my personal highlights of the aforementioned categories. Feel free to ask for more recommendations via twitter or in the comments. What are your own favorites?

The game screen has been built with a full motion video behind the HUD and the game layer.

The game screen as seen by the player internally consists of different layers. The top-most layer contains 2D graphics and displays information such as speed or remaining time. Behind this layer (or plate) the dynamic 3D game scene is being rendered. This layer contains the actual game elements the player controls. It has been built with OpenGL and displays the player’s character inside a rolling ball.

Now, in addition to this quite common setup we implemented a third layer in the back that plays a full motion video of a pre-rendered game scene. Even though this technique had been used for games such as Rebel Assault on the PC between the late 80’s and the early 90’s I am not aware of any iPhone game that integrates the game layer as tight as we did with matting and alpha compositing.

Game elements (a) with animated matting (b) appear inside (c) or even "behind" a background movie as illustrated on (d).

In no case we wanted to deliver a separate, affixed game layer above the movie. Instead, we tried to create the impression of embedded game elements (a) as an integrated part of the pre-rendered movie. With the help of an animated alpha mask (holdout matte) that corresponds to the wave movements of the river (b) we were able to let the water appear above the ball (c). And by using a matte with a true alpha channel we were even able to let submerged parts of the scene shine through the water surface as you can see on (d).

NIVEA FOR MEN WaterfallRace is available on the App Store for free and I will write another post including a screencast once the scheduled update for known bugs is out.

We were able to make this compositing feature work on the 1st generation iPod Touch and iPhone with the SDK 3.0. Please let me know if you are interested in the technical details on how to configure the EAGLView and MPMoviePlayerController or how to sync with the movie on the targeted iPhone OS 3.1. Also, please comment on this post if there’s another development aspect of this project you want to learn more about.

Update: Read the follow-up post about the NIVEA FOR MEN WaterfallRace.

Different functions to separate actual and displayed speed values

During our first phase of interaction tests we constantly got the feedback that the speedo was not accurate, though. Players had the impression to be way faster than the value displayed by the gauge. Others complained about how hard it was to gain speed in the lower region.

We decided to separate the actual speed used for game logic from the displayed value as shown by the speedometer. The paper “Rethinking the Progress Bar” I stumbled across a while ago when I was improving the user experience of a desktop application came in handy to do so. Chris Harrison et. al. discuss how different update rates of ordinary progress bars change the user’s perception of progress duration. They come up with some simple, yet useful functions to map a real progress value x to the displayed value f(x).

Selection of functions as described in Rethinking the Progress Bar

Whereas the results in the paper highlight functions that produce an accelerating progress bar in the end we needed to achieve the opposite for our game.

My speed is quite good by now but I am not at top speed, yet. I will try harder.

On slow speed ranges our speedometer simulated a higher speed and greater variation on speed changes. Beginners got immediate feedback and felt success that way. High speed values on the other hand were visually more difficult to achieve. That behavior seemed to motivate advanced players.

We first used the Inverse Fast Power function but eventually settled on Late Pause. No player ever mentioned a mismatch between the actual and the displayed value after our adjustments and nobody seemed to recognize the fake.

Links

• NIVEA FOR MEN WaterfallRace on the App Store
• “Rethinking the Progress Bar” by Chris Harrison, Brian Amento, Stacey Kuznetsov and Robert Bell
• itemis mobile application development

A week ago I stumbled across a small and free tool by Nicholas Zambetti called LiveView you can find on the App Store. Its concept is easy: On your Mac you run the LiveView ScreenCaster. That counter part renders an iPhone/iPad frame in its actual (huge!) dimensions on your monitor. Anything that shines through the screen of that imaginary device is being sent over the Wi-Fi. On the iPhone/iPad you can now run the LiveView app. It automatically finds every Mac currently running a ScreenCaster via Bonjour, asks you for the one to display and… Well, that’s it. You can now see your screen pixel perfect without exporting and importing images anymore.

Get a live copy of your screen on the iPad with the LiveView app.

As it turns out, this is just the way you can immediately test out anything you mock on your big screen. The ScreenCaster frame gives you the right idea about the actual dimensions of the device including its borders. You can adjust its position pixel by pixel with the cursor keys and finally press CMD+H to hide it. This is far more convenient than any VNC-based approach I tried before.

From there, work with Photoshop, Pixelmator, OmniGraffle or any other tool as usual. You can hide layers, resize elements, and change your font size while watching everything live on your device.

Impress your client and hand over the iPhone to your stunning audience while swapping layers. Or, use the tool as I did it with my current project at itemis. I looked at a flash website through the iPhone! That way, I was able to gain a first impression of what can be improved with a true iPhone app.

Unfortunately, the ScreenCaster cannot be rotated to send the data in landscape mode. Another flaw is the lack of a Windows version of the ScreenCaster. Nicholas, if you read this: I meant it the way I mailed it. Let me know if I can help you with the Windows port!

Links

• LiveView project website
• LiveView on the App Store
• blog post about LiveView by Gentry Underwood

The iPad after 15 minutes of usage

Weight and Typing

When I held this device in my hands this morning my first reaction was “oh, that’s heavy”. And in fact, after holding it a few minutes between my thumb and forefinger while swiping and touching through the apps this impression got stronger. Its weight becomes even more annoying while typing when you hold the device with one hand and type letter by letter with the other one.

“Not a problem” you might say, “one can place it on the table to type”. Well, yes and no. I was suprised how stable it was on the tabletop despite its curved flip side. The keys are way larger than those on the iPhone and typing with two hands would be a pleasure if there wasn’t the space bar. Each time I hit the spacer with my thumb the device slightly revolved around its own axis. Only cloth could prevent this.

“Why not using two thumbs for typing”, I thought by myself. This is what I do on the iPhone the whole time and it might work well on the iPad even though it’s larger. I was looking at the virtual keyboard to hit the proper keys with my thumbs and kept quite happy for a few letters. To conquer those larger distances with your fingers is one thing. To focus text and keyboard with my single pair of eyes simultanously simply doesn’t work. The keyboard at the bottom and text at the very top of the screen are so far away from each other on that huge display, this technique doesn’t work anymore. The seesaw between text and keyboard made me feel like I haven’t typed before at all.

In the end, only few postures on the sofa where you place the iPad on your lap worked fine for me.

Glossy Display

At first glance, this might only be an esthetic issue. Having finger prints all over the display is not really the end of the world. It’s just in the eye of the beholder. Right, but having reflections of your ceiling light, sunblinds or windows in general is more than annoying. When one finally found a way to type successfully on the iPad, there are not many options to prevent those reflections anymore. There’s no turnable display lid like the one on your laptop and you cannot easily change the angle between your tables and your windows – once you figured out how to type with two hands.

No Multi-User Support

I think of the iPad as the obvious extension of your living room. Your guests can look-up the soccer results, browse on youtube, pass it around to complete the pizza ordering or even check their facebook status. And the latter is exactly the problem with the iPad. As anything else in your living room the iPad is open to anyone. But what about your mails? What about your browser history and your stored logins? And what about your beloved calender, twitter, facebook or baking apps that grant access at a single finger tip?

Do you really have to lock the iPad? Is it just a toy and must not be used for serious data?

Conclusion

I am still fascinated by the iPad. It’s an amazing (some argue “magical”) piece of technology. I will get used to its weight and will learn how to type and look at the glossy display. But the lack of a “guest profile” or an equivalent way to hide away sensible data is something apple has to work on. Under these circumstances there’s only one option for me to use the iPad in future – I will need two of them.

100% Generated iPhone app of my EclipseCon talk

During my talk on Monday (slides on slideshare), I argued how you can leverage the ideas of MDSD to be more efficient when building iPhone apps with the help of domain-specific languages. To prove this, I built a running iPhone application in front of the audience in just 12 minutes. That application was able to browse the corporate blog of itemis as well as an index of itemis speakers and talks available at EclipseCon. From any screen you could jump to related bits of information. Everything was being loaded on demand and behaved like every other native application on the iPhone – since it is a native app. And the coolest thing: All this without leaving Eclipse!

As you can read in the interview I gave to The Bitsource I understand this particular demo app as an example. It represents an important part of many apps you can find in the stores today and the provided results are substitutional for the whole group of data-centric mobile applications itemis is going to deliver with the powerful help of MDSD.

Me holding the iPhone Mars Rover app. Click the image to watch a video of the iPhone Mars Rover app.

A rather unusual event we stumbled upon was the EclipseCon Mars Rover Challenge held in cooperation with the NASA JPL. It’s idea was to create the very best robotic control system to drive a robot across a prototypical Mars landscape. And guess what? Peter and I set out to deliver an iPhone application to do so.

To make a long and amusing story short, we came up with a vibrating and blinking iPhone app that allows you to steer the robot by tilting the device. Popular Science, Mashable, Slashdot and many other news sites caught up quickly – so we decided to publish a dedicated website (iPhoneMarsRover.com) discussing the details and providing more images on that particular application. At this point I want to thank Carsten and Michael for their invaluable help building that site. It’s a pleasure to work with such talented people.

And regarding the Mars Rover app: In future, we will most probably give droids the chance to hold that app in their hands – stay tuned!

Yes, itemis truly enters the market of mobile applications. And this goes way beyond ordinary iPhone, Android or Symbian apps. We are currently working on something that will knock your socks off! But that’s still a #secretproject…

Update 04/15/10: Dzone interviewed Peter and me about the aforementioned talk as well as the Mars Rover app. Read more details over there.

Links

• Slides of my MDSD on iPhone talk
• Video of the iPhone Mars Rover app
• iPhoneMarsRover.com
• Interview at BitSource about MDSD on iPhone
• Interview at DZone about MDSD on iPhone and Mars Rover

Nearly 200 attendees, 29 talks on 2 days and 3 winners of the best German iPhone App contest at the iPhone Developer Conference in Cologne

I had the chance to introduce the iPhonical project that applies model-driven techniques to produce parts of iPhone applications automatically. The presentation first describes a way to separate generated code from manually written source with the Objective-C concept of categories. From there, I demonstrated that even with powerful frameworks such as ObjectiveResource (an adapter to connect iPhone apps with Rails applications) one can benefit from code generation. With the help of the iPhonical DSL (implemented with Xtext) keeping data-centric classes in sync with the web application was way easier than coding them by hand.

The iPhoneDevCon was a very informative event. Many passionated developers or leads, growing studios and enthusiastic marketing representatives shared their ideas and during the breaks you could easily discuss different viewpoints of today’s opportunities and approaches.

Apart from the 3 winners of the “best German iPhone app” (according to iPhone & Co) a brand-new iPhone and Mac magazine, mac-developer, has been announced. As the name suggests it concentrates on developers and its first issue is quite promissing.

Links

• iPhone Developer Conference website
• Slides “iPhonical and model-driven software development for the iPhone” on SlideShare
• iPhonical project website
• mac-developer magazine (German)

My talk about model-driven iPhone development in general and the open source tool iPhonical in particular ranks among many other promising talks for developers.

Visit the German iPhoneDevCon in Cologne on December 01-02, 2009

If you are planning to attend please contact me in advance. Being a speaker I can give away a limited amount of promotion codes. This might safe you some expenses and allows you to buy me a drink in return

Under the hood, the animated ducky as well as the illusion of water is based on a hand-crafted physics engine. The water is implemented as a particle-based fluid simulation. Matthias Müller published a paper with the same title and great slides for SIGGRAPH 2007 as a starting point if you are interested in this topic. The ducky itself interacts as rigid body with these particles where buoyancy had been implemented explicitly. Play with Erik Neumann’s demo to have fun with rigid body physics. These concepts had been adjusted and combined with the accelerometer and touch sensors built into the iPhone to let the user interact with the simulation.

Different techniques had been applied to realize the illusion of a splashing ducky

Having just dots and circles as one could look at the physics engine was obviously not the ultimate goal. From the raw data of the simulation the water surface area had to be derived, the ducky had to be put in shape and some smooth animations and sounds were needed to round out the illusion of an interactive bathtub.

Without going into every detail I want to emphasize that the iPhone is not a MacBook Pro. Where you traditionally use marching cubes/squares to convert distinct particles into a cohesive area the processing power of mobile devices requires you to squeeze out every cycle by thinking outside the box and taking advantage of hardware acceleration. In this case, OpenGL ES offers a variety of techniques including framebuffers, blending functions and alpha tests to perform the needed steps by the GPU.

metaball technique from particles using blending and alpha functions

Some other findings during the project include the unreliability of sensor data as well as users’ unpredictable behavior. Be aware that humans and machines act differently under certain circumstances. For applications where the interaction between those both is crucial you should do usability tests early, often and extensively before delivery, again.

The iPhone is different from other mobile devices or the emulator. Generally, floating point arithmetic will be evaluated more efficient than fixed-point. Some GPU operations are executed faster on the device than on the emulator, others are of poor performance though. Therefore, you should profile on the device to test different approaches of your design regularly. And: You must not forget to do so with sound enabled since sound processing might take more than 30% of your overall processing power.

So, go ahead and grab your version of the Burlington Duck on the app store:

If you are interested in details (e.g. “unpredictable human factor”, production of the screencast, etc.), please let me know and use the comment function of this post.

OBJ2OPENGL does the latter and acts as a converter from model files to C/C++ headers that describe vertices of the faces, normals and texture coordinates as simple arrays of floats.

Texturized and lighted 3D model with 8,056 faces on the iPhone

OBJ2OPENGL is a Perl script that reads a Wavefront OBJ file describing a 3D object and writes a C/C++ include file describing the object in a form suitable for use with OpenGL ES. It is compatible with Objective C and the libraries of the iPhone SDK.

The original idea and code base of this script comes from Margaret Geroch who kindly allowed me to enhance and republish this version. It now supports texture mapping and stored normals (instead of recalculating them). I have changed the structure of the generated arrays to work with glDrawArrays instead of glDrawElements. This is particularly needed for normals and textures where a vertex holds different information for some shapes. Also, this version includes several command line options to better control its behavior (such as scaling and moving the origin or setting the object’s and output file’s name).

If you can go with the defaults the conversion is as simple as

./obj2opengl.pl banana.obj

To include the converted object all you have to do is

// include generated arrays
#import "banana.h"

// set input data to arrays
glVertexPointer(3, GL_FLOAT, 0, bananaVerts);
glNormalPointer(GL_FLOAT, 0, bananaNormals);
glTexCoordPointer(2, GL_FLOAT, 0, bananaTexCoords);

// draw data
glDrawArrays(GL_TRIANGLES, 0, bananaNumVerts);

Examples

These are two models and generated sample output that is included in the provided archive:

Downloads

• OBJ2OPENGL.ZIP (script and examples)
• OBJ2OPENGL.PL