New approach to photography? If you’ve heard of the Lytro, you probably know what I mean, but if not, here’s a quick explanation. Typical cameras capture a static image, measuring the intensity of light at millions of points (this applies to both digital and film if you think about it). Lytro does something a little different. In addition to measuring light intensity, they measure direction. Instead of storing points, they’re storing vectors. At least that’s how I understand it. By doing this, they’re able to store a more complete representation of the scene, something they call “light field photography”. The information is then used to reconstruct the scene on a screen in a technique reminiscent of ray tracing, the kind of thing that Pixar perfected. But because this is a photograph, it’s an accurate rendering, not the kind of cartoonish look that computer animation has. Anyway, by reconstructing the entire three dimensional space of the image and then determining where the focal plane is, they’re able to change the focus after you shoot the photograph. That’s revolutionary, and that’s why the Lytro is a big deal. (Harry McCracken makes the case that it’s the most revolutionary camera since the SX70.)

I actually did get my hands on a Lytro briefly back in December, though. I was walking through Bryant Park at lunch time around the corner from my office and stumbled across someone using a Lytro. I asked her if it was one, and of course it was. Turned out she was a project manager for Lytro. She let me take a few pictures with it, but since we didn’t have a computer handy, I didn’t get to see how they turned out. Still, it whet my appetite.

The packaging was attractive, once I got it out of the ugly cardboard box and removed the padding it was shipped with. I love how so many companies are taking notes from how Apple packages their products. With the Lytro, you open the box and the camera is just there, as if on a stage custom made for it. They clearly thought about the unboxing experience.

Right below the camera in the box was a small piece of paper, folded over several times. This is the quick start guide. There isn’t a manual; what serves as a manual is a few posts on Lytro’s support site. No matter, there’s not much to the camera’s interface. On the bottom, there’s a power button and a USB connector, and on the top there’s the shutter release and a very subtlely placed zoom control. That’s pretty much it. Now, I’m used to shooting with toy cameras, so I have no problem with a simple interface like this. But for gearheads, there’s a bit more to the interface; the LCD is a touch screen.

Lytro pushes the idea of “living pictures”, photographs that you can refocus dramatically as a form of storytelling. It’s an interesting concept not typically seen in still photography. I used to rack focus as a way of shifting perception a lot when I worked in television as a cameraman in college, so it’s not something completely foreign to me, but it’s very different from the way most photographs work. The technology behind the Lytro doesn’t dictate that this is the only way you can use light field photography, but that’s what their software supports at the moment. It’s kind of a one-trick pony right now. You need to compose your shots in a way that not everything is in the same focal plane, that is, something is close to the lens and something else is far away from the lens. Ideally, one of these somethings sheds light on the other of these somethings in a way that tells a story, or increases your understanding of the first something. If you’re not constructing your photos in this manner, then the photos can seem like little more than a gimmick.

There’s nothing inherent in the technology that causes this to be the case. One feature that’s missing from Lytro’s current software that they promise will come in a later release is the ability to focus everything in a scene. This changes the way you would use the camera dramatically. One feature of the camera is that the lens is at a constant f/2.0 aperture. That’s quite wide open, and in traditional cameras features a narrow depth of field. You can see this in the current software’s “living pictures”. That was a design choice Lytro made to make the living pictures more dramatic. But when the ability to focus everything becomes available, that use of f/2.0 for the lens means that you can use the camera in relatively low light and still capture an entire scene in focus. To do the equivalent with a traditional cameras would require stopping down the lens dramatically and using a concomitantly longer shutter speed. And one nice thing about that use case of the camera is that because you can focus everything in the scene later, you don’t have to worry about focusing it as you take the shot. Because of the way the camera works, there’s no auto-focus, and with that, no delay as the camera seeks the ideal focus. It’s just not necessary with the Lytro, and once the software is updated to enable wider depth of field, the camera will become more useful, not just for the limited case of the “living pictures”.

One thing I’ve seen some users ask for is the ability to more finely define depth-of-field. Without that, you’re getting a camera that effectively shoots at f/2.0 or f/32 and nothing in between. I hope Lytro takes these requests to heart.

One other possibility light field photography opens up is 3D photography. All the information needed to create a 3D rendition of your scenes is already contained in the files; it’s just a matter of Lytro programming their software to create it. They’ve promised to release this capacity sometime this year.

Something that would be possible, but that I haven’t seen Lytro promise, would be the ability to shift the focal plane, that is, turn the camera into a tilt-shift camera. This is a popular effect that causes your brain to interpret the photograph as a photo of miniatures of whatever it is you’re shooting. Most tilt-shift photographs on the web today are artificially generated, particularly because tilt-shift lenses can be wicked expensive. I’ve played with the effect a little by using one or another of the lenses from my Kiev 88cm with an adapter made by ARAX so that it fits on my Canon cameras. I’ve never quite gotten the hang of it, but I still try once in a while. If Lytro were to see fit to include this in their camera, it would provide another use case, one that can be pretty gimmicky in itself, but really no more so than the current “living picture” approach. I hope they do this, if not in the next release, then in one not long after the next release.

So does the camera work well? So long as you’re willing to work in the boundaries set by the “living picture” concept, yes, I think it does. It’s limiting, but no more so than using a Diana or Holga. Yes, it’s a lot more expensive than either of those cameras, but the Lytro has the promise of becoming more. I’ve shot just over a hundred photos with the camera since I got it on Friday. If you don’t take care to shoot within the constraints of the close/far rule, you get some pretty pedestrian looking shots. It can do some cool things, but right now, those cool things are all it can do.

I haven’t talked much about the act of actually shooting with the camera, though. The form factor of the camera is unusual, even unique. I’ve seen it described as looking like a boxy telescope. It’s definitely interesting. The shape is defined by what’s inside. The exterior is divided into two parts, one anodized aluminum (blue, graphite, or red, depending on which model you order), and one rubber. The metal section contains the lens, which is a zoom lens that covers the 35mm equivalent of 35mm to 280mm, all at f/2.0. At roughly the point where the metal meets the rubber is the light sensor, about which more later. Behind that is the board containing the electronics for the USB port, a lithium-ion battery, the main circuit board, and the LCD. The size of the LCD is defined by the size of the lens, and that’s unfortunate. It’s tiny. A more traditional form factor might have allowed for an LCD that did a better job of displaying what the camera is seeing, and for a larger interface. The touch interface of the LCD is clever; you can swipe up to get access to settings, including a switch to “creative” mode that allows you to focus right up to the lens instead of the 4-6 inches that “everyday” mode gives you (at the cost of some distant focus). In everyday mode, if you tap on something in the LCD before you take the picture, the camera changes the shutter speed to more accurately expose for that item. In creative mode, if you tap on something in the LCD before you take that picture, that’s where the median point for focus is set. You can tell which mode you’re in by looking at the LCD; if there’s a blue border around it, you’re in creative mode, otherwise you’re in everyday mode.

The camera turns on very quickly using either the power button or the shutter release, and because you don’t have to worry about focusing, you can go from instinct to shot in about a second in everyday mode. Unfortunately, if you’re outside in bright sunlight, you also don’t have to worry about framing, because it’s unlikely you’ll see much on the LCD, particularly if the sun is behind you. The LCD washes out much too easily. And if you’re a little off center of the LCD, the image posterizes and goes negative, which also makes it difficult to frame properly. That’s a shame, because the form factor is one that lends itself to shooting in a lot of different ways. I can definitely see holding the camera at foot level, for example, but it’s impossible to frame a shot taken in this way. Better LCDs are available; it’s a shame that Lytro didn’t spec one for the camera. As someone accustomed to shooting with toy cameras, where the preview is at best a rough approximation of what your shot will look like, this is perhaps less of an issue for me than for others, but it’s definitely something to consider.

Then there’s the elephant in the room: resolution. One reason Lytro positions the camera as being useful primarily for “living photos” is that you don’t print them; their natural habitat is on computers. The maximum resolution available is 1080 × 1080 pixels, or just over 1 megapixel. I don’t think you can even find a phone with resolution that low at this point. If you really want to print something, you might be okay at 5 × 5 inches, but you don’t want to go any higher than that. I exported a couple of photos as JPEGs; they weren’t anything I’d want to print. The camera really is optimized for online.

(In this image, for example, there’s something weird going on with the wall in the background. It’s as if the software put part of the wall in focus, then grabbed the rest of the wall from a completely different out of focus representation, rather than the gradual change in focus that I see in the Lytro desktop software.)

That said, if you understand how the camera works, it makes sense that the resolution is so low. The camera uses a traditional sensor inside, and its dimensions are 3280 × 3280 pixels, or about 10-11 megapixels (this is almost certainly the source of Lytro’s claim that they capture 11 megarays of information). In front of that sensor is a matrix of microlenses, each covering approximately 10 pixels square of the sensor to record the information the light field photograph requires. The information in each of those 10 × 10 squares goes up to make a single light field photograph. I can see the next generation of these cameras increasing the resolution to a more respectable 3 or 4 megapixels. Nokia just released a phone with a 41 megapixel sensor, and I’ve read of efforts to create 100 megapixel sensors for a reasonable price. For traditional photography, something like 100 megapixels is horrible overkill, but for light field photography? Could be just what the doctor ordered. But that’s about the next camera or the one after that, not about this camera.

If you want to see exactly how the Lytro stores information, International Man of Mystery Nirav Patel has released some software to do some basic manipulation of the LFP format that Lytro uses. The photos are stored by default in a Mac OS X package in your Photos directory, which means that individual files aren’t supposed to be accessible, but if you right-click on the Lytro file in your Photos directory and select Show Package Contents from the resulting menu, you’ll be able to get at the raw files, which live in the images directory and have the naming convention IMG-xxxx.LFP. And since Lytro uses a pretty standard sensor, they are pretty standard raw files, which means that once you extract the raw files using Patel’s software, you can use other software to open the raw file and see what Lytro is doing. Basically, you’ll see the 10 × 10 squares I mentioned (or maybe more accurately, hexagons). The magic that converts this into a light field photograph is contained in Lytro’s desktop software.

When you look in that directory, you’ll also see smaller files, with the naming convention IMG-xxxx-STK.LFP. These are the files that Lytro uploads to its servers rather than uploading the entire 16 MB full light field photograph. The file extension indicates that it’s the same format as the full photo, but that’s actually a bit of legerdemain; the file is simply a container for a number of JPEG files. That’s what’s uploaded to Lytro’s site; a number of JPEG files, each of which contains a focus state. I’m not sure how many they do for each picture; the one I looked at had five JPEGs, which seemed enough to cover the entire photo. For desktop browsers, the online viewer is implemented with Flash, but for devices without Flash (*cough* iOS, *cough*), the JPEG files are used without Flash to provide an equivalent experience. It’s a clever approach.

So is the camera worth getting? I’m not sending mine back. I like it, even with its shortcomings. You may feel differently, especially at this price point. It’s definitely a 1.0 product, and the limitations of the hardware indicate that there is a point beyond which the camera won’t go. No question that it will get better as Lytro releases software improvements, but in a couple of years, Lytro will be releasing cameras that blow this one away and are useful for a wider variety of purposes, even as a primary camera. If you get the camera, you’ll be a pioneer, and pioneers usually end up with arrows in their hats. But if you’re a pioneer, you get to have lots of fun exploring. Light field photography is a radical leap forward. It’s entirely possible that in ten years, all digital cameras will work this way, and the Lytro is your entry into that. It shouldn’t be your only camera, or even your primary camera. But it’s a trailblazer, and even if it doesn’t ultimately make it to the promised land, you can definitely see it from here.

Tags: photography plenoptic "light field photography" lytro revolutionary

4 comments

I watched the tutorials on the site, and it appears to have a well thought out interface. If you’ve used timeline-based animations in Flash, this won’t look like it’s from another planet. Beyond that, since I haven’t actually used the software, it’s hard to say, but my first impression on that front is positive.

Leaving aside the question of whether a plague of gratiuitous CSS animation is really an improvement on the long-standing plague of gratuitous Flash animations, how does the code the tool generates stand up?

Not well.

Looking at the examples on the product’s web site, we find that the typical source code of a Hype-generated site looks something like this:

There is almost no HTML5 in your HTML5 document. All of the content you’ve so laboriously animated? That lives in a Javascript created by the tool. When Teh Googlebot comes to visit, it will see the following on a Hype-generated web site:

In other words, you won’t show up in Google at all. Further, if you need to update your site and you’re not near the computer where you created your Hype site, you’re not going to be able to edit the code by hand to fix that glaring typo that one of your customers called out to you on the trade show floor. Not that you would want to; the HTML that they’re generating inside that Javascript is a mess, a mix of divs, spans, and (OMG!) font tags, with nary a header tag in sight. So even if they were generating actual HTML, you probably wouldn’t want to use it, because Google would still find it a mess to parse. Years of research into search engine optimization are completely ignored here.

All that CSS that’s making it easier to update your site comprehensively? It’s all inline, totally defeating the purpose of CSS. If you want to incorporate some of the effects generated by Hype into your existing site, good luck. Hype appears to want to own your whole site.

But hey, you’ll get some whizzy effects, particularly in modern browsers.

I’m not averse to whizzy effects. And I figure any technological advance like this is going to be abused initially, just as early desktop publishers used way too many fonts at first, and as Flash developers made long, large, pointless splash screen animations and then tried to put a band-aid on their creations with the addition of the Skip Intro link. It’s part of figuring out what works, and that’s fine. Go wild. Make art. But not with Hype. Not if you’re trying to actually accomplish something with your web site.

Color me unimpressed.

Tags: hype

No comments (yet)

The biggest problem is that the HTML/CSS/JavaScript technology stack on these devices is slower than natively-executed code. It’s a lot slower. This is the case whether you’re targeting Mobile Safari or using an app shell like PhoneGap, which Faruk mentions but says he hasn’t had a chance to play with yet. I have played with it for personal projects. The app I worked on at work was a custom hybrid app, with a crunchy Objective-C based shell surrounding chewy HTML, CSS and JavaScript. The idea was to have the shell simply provide a UIWebView window, and all the interaction would be performed within that by the JavaScript. It was incredibly difficult to get those interactions to act the way they would if they were executed by natively compiled code. There are a number of tricks you can use, taking advantage of the fact that 3D CSS transforms and opacity are hardware accelerated, but that introduced other complications, particularly around the limited memory available on the first generation iPad.

The tools available to debug your JavaScript are primitive at best. In Mobile Safari, there’s the debug console, but you can’t set breakpoints and step through the code and examine the variables to determine why things are breaking. In a hybrid app using UIWebView, you don’t even have that. We wound up hacking something together where a JavaScript library would URL encode anything you wanted to log, then use it to create and call a URL with a custom URI scheme, which the app shell would then interpret, understand that it was a log message, and output it into the XCode debugging window. My partner on the project, who wrote the Objective-C shell, had difficulty with the memory barriers we kept running into, as XCode doesn’t provide much information about what’s happening inside the UIWebView.

Another problem we ran into is that UIWebView didn’t provide support for accessing orientation in JavaScript. The window.orientation object existed, but wasn’t updated when the screen changed; it always reported that the screen was at 0 degrees, if I recall correctly. My colleague had to provide a way for us to access that information as part of the Objective-C shell that he wrote. I found that a really puzzling omission on Apple’s part. Note that Mobile Safari does update window.orientation properly. I think PhoneGap might also have implemented support for this to make up for Apple’s omission.

The worst thing was that implementing the entire interface in HTML/CSS/Javascript wound up creating an experience where items flickered when you moved them. Drop shadows and backgrounds proved particularly problematic. Matteo Spinelli, the author of the incredibly useful but sometimes flaky iScroll, has a post about how to avoid flickering by applying hardware accelerated transforms to items that flicker, but we found that then you exacerbate the memory limitations of UIWebView.

My colleague and I discussed this extensively. I’ve come to believe that there’s a place for UIWebView in App Store apps like this, but that implementing the entire interface within one is a bad idea. The user experience just isn’t up to the standard of a native app. And if you’re distributing your app through the App Store, then the expectation is that your app will behave just like a native app. The user doesn’t know how you implemented the app they just downloaded. They just expect it to work like every other app they’ve downloaded. Unless you have a barebones interface, your hybrid app is unlikely to provide that experience.

I’ve looked at a lot of apps on the iPad that distribute periodic content (like magazines and other news organizations) where the ability to use the HTML/CSS/JavaScript tech stack for periodic production tasks is a huge benefit. Very few of them seem to be implementing the entire interface with UIWebViews. The BBC News iPad app looks likely to be implementing some part of the app with UIWebView, specifically the display of articles. My colleague and I discussed how it’s much easier to display content on the screen using a UIWebView rather than from native code. But I don’t think the entire interface is done that way, or specifically, not all at once. It may be true that everything on the screen is drawn by a UIWebView, but it looks to me like the screen is carved up into several of them. I think the same could be said about the NPR News iPad app. Other apps seem to get around problems with rendering pages by prerendering them, which is to say, they’re showing you pictures of words. Any app that’s built around the Adobe Digital Publishing Platform is using this approach (I’m not sure if they're using UIWebView or native calls to display the graphics, though). This bloats file sizes hugely, as can be seen by downloading an issue of Wired magazine from the App Store. And I think it’s probably inaccessible; VoiceOver doesn’t do text recognition in pictures.

There’s definitely a place for the hybrid workflow, but you’re not going to be able to create an identical experience with web technologies that you can with native code. Integrating a web technology-based workflow into an app requires a lot of thought, and particularly careful design that takes the limitations of the targeted device(s) into account. Throughout the development process, I kept noting that the iPad was definitely a 1.0 device, despite its iOS 3.2 pedigree.

User expectations are different for a web-based app. The experience of using a browser is one where network latency is common and users are not surprised to find that pages take a while to fully compose themselves on the screen. This is even more the case when you’re using an iPad over a 3G connection, combining the relatively slow processor with the relatively slow network connection. So the kind of interface anomalies you might see because of the relative slowness of UIWebView (as opposed to native apps) are more acceptable. You can, and should, make sure you optimize the experience by taking advantage of hardware acceleration where you can, but if things take a half second to redraw, users are more likely to find that acceptable in the context of using a mobile browser. They see that all the time. There are apps where that kind of interaction is acceptable, and that would be a good case for creating a mobile-enhanced web site/app rather than a native app. But that kind of app is terribly limited.

Apple initially tried to tell developers that this would be the way that all “apps” for the iPhone would be created. Very little happened; some people tried to follow Apple’s direction, others went down the jailbreaking path. Neither resulted in a flood of applications. Once Apple delivered an SDK that enabled developers to take full advantage of the power of the device, we saw an explosion of apps that changed the way people use their phones.

One of the things that interested me most about the recent iPad 2/iOS 4.3 announcements was that Apple had ported its new-generation Nitro JavaScript engine to Mobile Safari (and presumably to UIWebView). Honestly, I was surprised that it hadn’t been there from the start, although its absence explains a lot of what I saw. It’s possible that these limitations will disappear as the hardware gets more powerful. But it’s also possible that as the hardware gets more powerful, native apps will blaze the trail due to their inherent relative speed when it comes to creating new and amazing apps that just wouldn’t be possible with the HTML/CSS/JS tech stack until a couple more hardware and/or software revisions down the road.

So by all means, if you have an idea that would make a good app and you think it could be implemented either as a web application or as a hybrid application, jump right in, but make sure you’re aware of the limitations of the approach. You’re very unlikely to replicate the native app experience from a web application, and in the hybrid approach, it’s going to be a lot harder than you thought.

Tags: faruk ateş iPad mobile safari UIWebView

No comments (yet)

Sadly, that led us to this video by the Jacob Sisters:

Truly irritaining. And number 1 in Canada!

Tags: jacob sisters

1 comment

The Clean seemed a little off last night. I've seen them probably four times over the years, and this was easily the worst. That said, much like sex, bad Clean is better than good anything else. I don't know what the problem was. It might have been a bad mix; The Clean is a sturdy platform put up by Robert Scott (bass) and Hamish Kilgour (drums) for David Kilgour (guitars and organ) to build mountains of sound on. But there were a lot of points at which you could hardly hear David's guitar, at least where I was standing. Even when you could, it often lacked the bite of Kilgour's best guitar work. Borrowed guitar and amp, maybe? Or maybe the band's just Getting Older. I never expect them to be tight; the ramshackle nature of their performances is expected given that one member lives in New York and the other two in New Zealand, and usually it's charming. It didn't bother me that songs kind of fell apart at the end rather than coming to a full stop all at once. But something was just off last night. Even the encore closer, "Point That Thing Somewhere Else", wasn't the usual rouser that it is. I know the band likes to mess with their songs; I even have a copy of "Point That Thing" where the main melody is done on piano, and the bite of the song survives that. Even with all that, it was a solid B perfomance.

Vermillion Sands, who opened, were a band I'd never heard of. I thought they did a good job winning over the crowd with their catchy songs. I bought their CD from their singer after the show, so I must have thought they were pretty good. They'd fit right in on Little Steven's Underground Garage. B+/A-.

I was dragging this morning, but made it through the day without falling asleep, so I guess I'm not as old as I felt at 6:53 this morning when I woke up (53 minutes late and I still made the last express train into the city and got to work at my usual time).

Tags: maxwells the clean vermillion sands dump hoboken

No comments (yet)

The monument took the form of a stylized rendition of the original, which you can see in both of the links in the first paragraph.

Sadly, Bell Labs moved out of the building in Holmdel a few years ago. There have been attempts to sell the building and redevelop the huge tract of land in the middle of a very wealthy community, but they've come to naught, and the property is blocked off and basically inaccessible at this point. Back in 2007, shortly before the barriers were erected, I visited the site and took some pictures. Lucent, the corporate father of Bell Labs, had fallen on hard times, and the maintenance of a monument to their glorious corporate and scientific past at a location they didn't even want to own any more took a pretty low priority.

The text of the monument reads:

At this location in 1931, Karl Jansky, a Bell Laboratories physicist and radio researcher, recorded for the first time radio signals from beyond the Earth. The source of these signals -- radio noise at a wavelength of 14.6 meters -- was the center of our Milky Way galaxy.

This sculpture commemorates Jansky's discovery, first announced in 1933, which gave birth to the science of radio astronomy. The sculpture is oriented as Jansky’s antenna was at 7:10 p.m. on September 16, 1932, at a moment of maximum signal. As his directional antenna rotated, the center of our galaxy came into view in the direction of the constellation Sagittarius, low on the southern horizon.

Radio Astronomy pioneer Karl Jansky died in 1950, years before the scientific community realized the significance of his discovery. In 1973, the International Astronomical Union gave his name to the international unit of radio flux density. Jansky’s work led to a number of breakthroughs in astronomy: the discovery of quasars, pulsars, radio galaxies, and near this site in 1964, the Nobel Prize-winning discovery by Bell Laboratories scientists of the cosmic microwave background which has revolutionized our understanding of the origin of the universe.

It’s sad that the monument was left to the elements and blocked by barricades. I hope that someday it will be accessible again.

Tags: jansky

3 comments

...always makes me think of the video Chris Knox and Alec Bathgate did for the Tall Dwarfs song Turning Brown and Torn in Two

That’s only fitting, because Little Girls kind of sound like they could have been a Flying Nun band.

Tags: tall dwarfs little girls

No comments (yet)

Uh-uh.

Think more along the lines of classic NZ noisemeisters Bailter Space, shimmering waves of melodic noise. Great stuff. In this video from KEXP, they take Big Black's classic “Racer X” and make it their own.

They’re currently touring the world and releasing a single every two months that includes an interesting cover on the B-side. The first one, which had Racer X on it, is sold out, but you can get their new Younger Us single b/w a cover of X’s “Sex and Dying in High Society”. While you’re there, feel free to buy their two albums as well. Great stuff.

Tags: japandroids bigblack racerx

3 comments

Tags: music ponies laurabarrett

2 comments

Tags: pinhole communism kiev polaroid photography

No comments (yet)