Mobile web app fail

As an experiment I wanted to see if a web app could be created that would send images from a wi-fi enabled mobile device to a computer. The reason was that the computer could easily serve the web app directly to a user's mobile device and the user could then upload their images to the computer and both operations could be done without needing to download a native app from an application vendor or use a data plan.

The proof-of-concept requirements were fairly broad and simple: serve the necessary files that make up the web app from the computer, allow the user to choose image files, and upload the images to the computer. Serving the files was simple, I created a basic stand-alone server process that could respond to requests for files and also handle the image files that were uploaded to it via XHRs from the device's browser. The user requested the start page by scanning a QR code that launched the browser and navigated to the app's URL. Problems really started with the web browsers available on mobile devices.

The largest problem was the iOS web browser. Prior to recently released iOS 6 the iOS browser didn't allow the user to choose files for upload. The basic HTML "input" element with type "file" was not supported in the browser. It seems hard to believe this was an unintentional oversight on Apple's part. The practical effect was that this forced developers to create native iOS apps rather than allowing them to create a single web app for any mobile device. There will remain a large number of devices running iOS 5 and earlier versions of iOS for years to come. As developers we will be living with a crippled iOS browser for a long time. IE 6 anyone?

The next problem involved displaying thumbnails of multiple images. On my Nexus S phone both the native and Chrome browsers crashed after opening multiple images. Looking at the logs I found that the browsers were running out of memory. I was able to improve the situation considerably, but not entirely fix it, by setting the img element's width and height attributes to the size of the image that actually displayed rather than relying on CSS.

Once that problem was minimized I was able to select images, insert the thumbnails into the DOM and then upload the the files to the computer. Some positives from this experiment: all of the Android browsers had partial support for the File API (notable exception below), data URIs could be used to access the image information in the files, and finally @font-face support so I could use Font Awesome for the images on buttons and menus. Negatives: while the File API's File object is supported Android doesn't have multi-file select functionality for browsers. Each image file had to be chosen one at a time from the Gallery application which became tedious by the second file selection.

I'd like to mention that of the three browsers I used during development only Mozilla's Firefox browser for Android was able to reliably display and upload files. Unfortunately for the proof-of-concept web app it had to run on at least the iOS and Android native browsers. Other browsers like Firefox or Dolphin just don't have a large enough market share to make them an acceptable solution.

Having said that, my experience with the shortcomings of many mobile web browsers has reinforced my belief that Mozilla's mobile browser and Firefox mobile operating system are as important now as Firefox was ten years ago in helping to push innovation forward, especially on mobile devices, and to keep the web open.

Lastly a tool that was indispensable and made development easier and faster was desktop Firefox's Responsive Design View mode. In this mode (available under the "Web Developer" menu) the browser's viewport is resized to emulate various common monitor and mobile device screen dimensions or you can set custom dimensions as needed. Using this I was able to set the browser to the same size as my phone and do most of the development work on my desktop, only using the phone to periodically test functionality. If you need to design a web page that is "responsive" to different size screens or screens that change size, like switching from portrait to landscape then this tool is a must have.

Set an Android ringtone

Android has a lot of quirks and annoyances, a common one is setting a custom ringtone, then later no ringtone plays when there is an incoming call (the phone is silent). This happens when the audio file is not located in the correct file location or the wrong application is used when choosing an audio file as a ringtone. Later if the phone is plugged into a computer and the USB file system is enabled the custom ringtone becomes unavailable and the OS "forgets" the ringtone. This is how to use and set a custom ringtone from an mp3, ogg, etc. audio file so that the phone won't forget:

1. Connect your phone to a PC and copy the file to the folder \sdcard\media\audio\ringtones on your phone. You must create any of these folders that don't exist.
2. Disconnect your phone from the PC.
3. Select a contact and from the menu choose "Set ringtone". If given the option to choose an application to select the ringtone choose "Media Storage" and then select the "Always" button. Never use any application other than "Media Storage" to choose the audio file.
4. From the list of available media choose your ringtone (the name that appears depends on the IDv3 tags in the audio file you added). But wait! What if you don't see the file you just added? The media list is populated by the OS scanning periodically for audio files and it can take some time to discover the new file. So you have two options: first is to just wait a while and retry until the file shows up in the list, second you can clear the media cache and force it to rebuild which still means waiting a while. To clear the cache do this: menu - Manage Apps - All - Media Storage - Clear data. Now reboot the phone and wait for Media Storage data to be rebuilt. This can take tens of minutes. Eventually your audio file will show up in the list.

You can also set the phone's default ringtone by selecting System settings - Sound - Phone ringtone in place of step 3. That's it, now your ringtones should always be available even after treating the phone as USB storage.

When Canon G1X video and Sony Vegas collide

Do you end up with periodic black frames followed by bright ones when editing video from Canon's G1X camera using Sony Vegas Movie Studio 10? If so try re-encoding the camera generated MOV files using Huffyuv (which is lossless) before editing them in Vegas.

Canon's G1X camera is capable of capturing H.264 encoded video and storing it in an MOV file. The video captured is 1920x1080 at 23.976 frames per second (you can read about this not quite 24fps if you're interested).

I had no problem opening the video files in Sony Vegas and working with them in any of the tools. For my initial test I simply stacked up a few short clips directly from the camera and started the render to H.264 process. In playback the first minute or so of the rendered video was fine but eventually a black frame was inserted into the middle of a clip, not at a transition, and bright frames followed it for a second or so before normal brightness slowly returned. Over the course of my five minute video this happened several times.

I decided to avoid using the H.264 encoded camera input by using ffmpeg to convert the camera video to Huffyuv encoded video since this would give me lossless video to experiment with. (And now a little detour getting the Huffyuv codec installed on my Windows 7 system so that it would be available for Vegas.) Once that was done I opened the Huffyuv video using Vegas in the same order that I had done before with the camera video and rendered to H.264. This time the resulting video played back without any defects.

It appears to me that Sony Vegas is not really capable of handling the 23.976 frame rate properly when working with H.264 encoded 1920x1080 input and periodically inserts a black frame during the output render. When the output is then H.264 encoded the black frame affects the compression of following frames, making them brighter.

The summary: Canon G1X video should be converted to a lossless encoding (Huffyuv works fine) before being edited on Sony Vegas 10. Other versions of Vegas may or may not need the same treatment.

Synchronizing multiple jQuery ajax calls using the when function

There is a lot of information about jQuery's ajax() function the jqXHR object and using jqXHR as a Deferred object with the when() function, but I couldn't find any examples that illustrated all this functionality working together.

The problem I solved using ajax() and when() is not unique or unusual, I had an unknown number of simultaneous ajax() calls to make and wanted code execution to continue on a single callback function when they had all completed. This is exactly the type of situation that Deferred objects and the when() function were created to handle. If you are used to working in multi-threaded environments you can think of when() as a kind of synchronization object. You can pass multiple jqXHR objects, returned by ajax() functions, to when(). You can chain a then() or done() function to the Deferred object that when() returns and the callback function you pass as an argument to the chained function will be invoked after all of the ajax() calls have completed.

Something I wanted to do to solve my problem but didn't know how to do, or even if it was possible, was to pass an unknown number of jqXHR objects to when() and get back the results for each jqXHR. One of the strange and wonderful things about javascript is that functions are objects and as objects they can, and do, have their own functions! Another peculiarity is that Javascript functions will take any number of arguments, regardless of how the function is originally defined, this matters because it means when() can take any number of Deferred objects as parameters. If you know each of the Deferred objects ahead of time you can simply pass them to when() as parameters (e.g. $.when(deferred1, deferred2)). If you don't know the number of objects you can gather them together in an array and pass it to when() using the apply function. The apply function takes an array of objects and passes them to its owner function as a set of parameters (be warned that the number of parameters a function can accept is going to be limited by the javascript environment, don't go crazy).

Now I knew how to pass an unknown number of jqXHR objects to when(). How do I get the results of the ajax() calls? The when() function returned a Deferred object and I passed its done() function a callback function to invoke when all the ajax() calls were, well, done. I couldn't find any examples of what this callback function should look like when an array of Deferred objects were passed to when(). My initial thought was that an array of result objects would be passed to the callback, but the single argument I defined only had the result of the first ajax() call. After a long time of experimentation, fruitless searching, and head scratching I suddenly remembered the arguments local variable that is available in every function. I realized that my callback function was being passed a result object as a separate parameter for each ajax() call. Sure enough I found that the length of the callback function's arguments matched the number of jqXHR objects passed to when() and that iterating over it I could get a result object for each call.

So finally here is a code snippet illustrating how to pass an array of jqXHR objects to when() and get back the results! It's easier to look at the original Gist or you can try it out in jsfiddle to see it work.

One final technical note: the jqXHR is not really a Deferred object, it implements the Promise interface but for the sake of simplicity I called it a Deferred object.

Facebook Graph API tweaking: fields

Want to improve the performance of your Facebook Graph API calls? Try trimming your requests down to just the information you need. During recent optimization and testing of some Facebook Graph API client code my colleague Bob determined just how much time could be saved by tuning the API requests. Typical requests result in the return of a default object, for example this URL

https://graph.facebook.com/6547384565867889

returns this album object:

{
  "id": "6547384565867889",
  "from": {
    "name": "John Doe",
    "id": "9834439837"
  }, 
  "name": "Mobile Uploads", 
  "link": "https://www.facebook.com/album.php?fbid=6547384565867889&id=9834439837&aid=743987489", 
  "cover_photo": "748397943639", 
  "privacy": "custom", 
  "count": 1, 
  "type": "album", 
  "created_time": "2011-11-24T16:04:54+0000", 
  "updated_time": "2011-11-24T16:04:55+0000", 
  "can_upload": false
}

The client application doesn't need most of this information, all it needs are the id, name, photo count, and when it was created. By adding the optional fields parameter with a comma separated list of field names to the request:

https://graph.facebook.com/6547384565867889?fields=id,name,count,created_time

an object with only the requested information is returned (apparently Facebook gives us "type" for free):

{
  "id": "6547384565867889", 
  "name": "Mobile Uploads", 
  "count": 1, 
  "created_time": "2011-11-24T16:04:54+0000", 
  "type": "album"
}

Not only does this result in less information being transmitted to the client, more importantly it results in considerably shorter response times from Facebook. It seems that retrieving this information requires significant lookup effort on Facebook's servers and asking for less information means less rummaging through the datastore for all the bits. Bob found that getting 57 albums containing 2,434 photos from his account using the default request took 90 seconds. After adding the fields parameter with only the fields required it took only 40 seconds, less than half the original time! Of course YMMV based on the network, we also found that eliminating likes and comments had the largest effect in reducing response time. If you are working on an application that gets large amounts of data from Facebook it may be worth the effort to consider what information is being provided and only get what the client needs.

Facebook client-side authorization

"OAuth"

This one word can make software developers tremble in fear. Add "Facebook" to it and those same developers start sliding the monitors on their desks together to hide behind, like pioneers circling the wagons at night.

Granted things are better now than they used to be; OAuth 2.0 and the Facebook Graph API are much easier to work with than the old REST API with OAuth 1.0 (Remember signatures? No? You're lucky). However authorizing an app or logging in with Facebook can still be a challenge. I really wanted to be able authenticate as a Facebook app in a single HTML file with javascript interspersed just to see if it could be done in one file. What I've found is, that can't be done, but something almost like it can be.

Originally I had one page that would navigate based on the current Facebook login state which was determined by the parameters passed to the page. Roughly the Facebook authorization workflow was:

1. App page initially displayed in a Facebook iframe: no access_token parameter.
2. Navigate the browser window (not the iframe) to the OAuth dialog. Pass the page URL as the redirect parameter.
3. User authorizes / logs in.
4. App page is now displayed in the browser window with the access_token (remember we left the Facebook iframe when navigating to the oauth dialog), navigate to the app's Facebook canvas page and include the access_token in the canvas page URL.
5. App page is displayed in the Facebook iframe and now it has the access_token.

This could work accept the Chrome browser only allows two redirects and this scheme requires three. After flailing with various fruitless searches ("facebook" + "javascript" + "login" does not yield a lot of helpful hits) I found a cached copy of this page that had the missing piece I needed, it used the Facebook Javascript SDK to determine if a user was logged in or not. I changed my code to use this method and then navigate to the oauth dialog or launch the app based on the result. This is the final workflow:

1. App page initially displayed in a Facebook iframe: app not authorized / user not logged in.
2. Navigate the browser window (not the iframe) to the OAuth dialog. Pass the app canvas page URL (not the URL of the HTML file) as the redirect parameter.
3. User authorizes / logs in.
4. App page is displayed in the Facebook iframe: app authorized / user logged in. Launch the app.

So there you go, authorizing an app and logging into Facebook done entirely client-side with one HTML file that includes the Facebook Javacript SDK (full disclosure: I also included jQuery for convenience though the code could be rewritten to avoid using it). You can try this code or use it by getting the source from this github gist (easy) or copy and paste it from the horribly clipped embedded view below (hard).

Animated menu for a web page

I'm working on revamping Smashingline and wanted to make the menu at the top a little more interesting. I decided to create a menu with two "pointers." One to point at the item your cursor is currently over (highlighted) and another to indicate the current item (selected). The menu was made with HTML5, Javascript, and CSS. I also used jQuery to animate various elements. I was inspired by the page eCSspert to use only HTML and CSS for the graphic elements.

Main

Search

Contact

Essentially the menu consists of a "menu bar" div with "menu item" divs above it. When the cursor passes over the bar its mouseover event is fired and the highlight pointer is shown. When the cursor moves from one menu item to the next the animation is triggered causing the pointer to move. One of the problems with this approach is that when transitioning from one item div to another the bar div fires a mouseout event even though the location of the cursor is still "inside" the bar div. Event propagation is probably one of the most common problems when working with mouseenter and mouseout events. Fortunately jQuery has solved this problem by implementing its own mouseleave event which was introduced by Microsoft and is supported natively in IE. The mouseleave event is only fired when the cursor actually exits the region of an element, no matter the number of child elements. Very handy.

The pointers are constructed from HTML div elements and CSS. Generally each of the two pointers has a base div that is the same width as the menu items. The div's height and setting its overflow property to hidden dictates how much of child div which is rotated 45 degrees is visible.

Finally since you may want the web page to take some action once the user makes a choice a callback can be set on the MenuMarker class. This function is invoked once the animation is finished. The demo here displays an annoying alert dialog with the the text from the menu item displayed.

The source code (in three separate files) is available for download from a Github Gist. These can also be combined into a single HTML file and loaded dynamically as needed using the jQuery load method.