Sunday, July 6, 2008

Snow Leopard - Apple Next OS X ?

Apple’s public introduction of Snow Leopard, the next version of Mac OS X, was decidedly brief at WWDC, with only passing public mention of its new feature set. That’s in part because the company is delivering something nearly unheard of in the consumer software industry: Apple is advancing a new software product that improves upon its fundamentals rather than advancing a lot of marketing features.

Nothing New, Everything Newer.

As anticipated by multiple reports, Snow Leopard will not advance any major new consumer-facing marketing features apart from new support for Microsoft’s Exchange Server 2007 push messaging in its new versions of Mail, Address Book, and iCal. That move will match the functionality outlined for the iPhone 2.0 software update on the Mac desktop.

The fact that Apple is building its own Exchange Server support in both Mac OS X and the iPhone indicates that the company will no longer be waiting around reliant on Microsoft to make Macs fit into corporate environments with the dreadful embarrassment that is Entourage, just as I surmised earlier. Apple is taking matters into its own hands.

Additionally, by delivering Exchange support itself Apple will be also be able to offer its own alternatives to Exchange in parallel, much the same as how Microsoft leveraged its Mac Office software to develop Windows as an alternative to the Mac OS. This time around however, its Microsoft’s technology that will be routed around and replaced.

Apple’s first Exchange alternative is Mobile Me, a subscription based web service that serves as the next generation of its existing .Mac service. Beyond Mobile Me, Apple will also be offering an enhanced version of its own server product for businesses interested in hosting their own Exchange alternative.

Trickle Down Tech: iPhone Software and the Mac Desktop.

In addition to inheriting the iPhone’s support for Exchange, the Mac OS X desktop is benefiting from investments made for iPhone in other respects as well. Last year, Leopard’s Core Animation sprang from work developed for the iPhone. Originally called the Layer Kit, Core Animation was conceived in order to give the iPhone a slick UI. Layer Kit was essentially a scaled down, mobile version of the desktop window server with new support for a highly animated new user interface explicitly designed to be easy for developers to fully exploit. Apple subsequently reused the technology in Leopard to make it just as easy to add rich interface polish to desktop apps as well.

In Snow Leopard, there will be another example of trickle down tech from the iPhone: QuickTime X (as in ten, not ex). Apple already performed a major overhaul of its flagship media architecture in QuickTime 7, adding support for modern audio and video codecs and ridding it of a lot of old legacy from the media architecture’s early 90s origins. However, QuickTime is still a complex assortment of software components, designed to work with media in nearly any codec imaginable and support everything from simple playback to complex media authoring. It is essentially an operating system for media.

When Apple developed media services for the iPhone, it started fresh with a pared down set of objectives. The iPhone is designed only to play music and video; it doesn’t need to edit or author it. Further, the iPhone uses specialized hardware that allows it to efficiently decompress H.264 video and AAC or MP3 audio; it doesn’t need to play any random files users can scrounge up on the web in archaic or non-standard formats. By limiting the iPhone to playback of modern codecs, Apple could create a really tight, highly efficient subset of QuickTime that performed well on mobile hardware while being conservative with its battery use.

Snow Leopard will make use of that same mobile-optimized playback software when playing any media that uses modern codecs. That gives desktop Macs the same highly efficient playback performance while still allowing them to fall back to the standard QuickTime routines when playing older codecs or anything requiring proprietary plugins. The marketing name for this iPhone-derived boost is QuickTime X.

The Pristine Technology Stack.

Adding marketing features has complicated the performance of Mac OS X. Even though each new reference release has typically boosted the overall speed of existing Macs, the potential for things ever getting too fast has been tempered by demands the new features in each release have required.

Tiger introduced Spotlight searching which while relatively efficient for desktop search has added a significant amount of overhead. Tiger’s Dashboard similarly delivered a far less demanding architecture for running applets than Konfabulator, but it still adds some extra work for the operating system to manage that users see as a hit against overall performance.

Leopard’s Time Machine is similarly designed to back up files at reasonable intervals and avoid wasting too many cycles on iterating through files manually to see what needs to be backed up, but it still has the potential to hammer performance while doing its thing.

By spending a full release cycle on tuning up existing services rather than tacking on more features, Apple will provide users with far more value in the long run, and be well positioned to eviscerate Windows 7 when it arrives in the ballpark of 2010, likely around the same time Apple will be showing off the release of Mac OS X 10.7.

Handling Processes like Network Packets: Grand Central.
Of course, part of the reason Mac OS X gets faster in each release while tacking on those “200 to 300 new features” (including the more ambitious undertakings such as Spotlight and Dashboard) is that Apple has also been working on its fundamentals all this time, too.

Throughout the development of Mac OS X, Apple has reexamined the old ways of doing things in UNIX and proposed new architectures. One example is launchd, the process that manages the launching, termination, and supervision of other processes in the system. It replaces a variety of existing process managers including init, rc, inetd, xinetd, atd, crond and watchdogd. Few UNIX vendors would bother to engineer an entirely new way to do things, and if undertaken in the FOSS world, such an innovation would rarely be adopted by enough of the Linux community to ever matter.

Apple now has the resources to create its own weather, so it can fix outstanding problems and then reap the benefits of that investment, frequently across multiple product offerings. The latest example of this is Grand Central, a new thread management architecture that greatly simplifies developers’ ability to take advantage of the multiple cores now being used in modern CPUs, as well as the raw processing power available in GPUs (graphical processor units) on the system’s video card.

Rather than expecting each developer to become an expert in the black art of multithreading, Apple has built sophisticated process management into the kernel where it belongs and added language conventions that enable mere mortals to take advantage of a wide variety of different hardware that users might have at their disposal.

Grand Central Dispatch manages processes in a manner analogous to modern networking. Old telephone equipment used to use circuit switching to transmit information over networks; a dedicated circuit path is easy to set up but it is also expensive and potentially fragile. Modern networking uses packet switching, which breaks up data, phone conversations, or video streams into packets and routes each of them independently in a far more efficient way that is also resilient to network outages. Packets get routed around the problems.

Snow Leopard’s Grand Central Dispatch does the same thing for processes, packetizing tasks into Blocks and routing them to available processing cores as efficiently as possible. It can also manage the big picture for the whole system, adjusting how it balances its tasks as the performance load increases. This would be close to impossible for Individual developers to do themselves.

More Bits of Computing: 64 And OpenCL.

It shouldn’t be surprising that Snow Leopard extends support for 64-bit processing, which became possible in Tiger and went mainstream in Leopard. Apple will be making the kernel and nearly all of its user apps 64-bit savvy, and is making it easy for developers to do the same for their own software.

The potential downside to 64-bit computing is that many processes will consume more memory to do the same thing. The upside is a significant, across the board performance improvement. For certain processor intensive apps, moving to 64-bits makes a massive improvement. These both offset the increase in memory needed, just as computers begin running into the 4 GB limit of 32-bit architectures. The move to 64-bit support system wide will allow Macs to eventually load up on an insane 16 terabytes of RAM, but in the shorter term will make better use of the 4 GB in the latest Mac Book Pros and the 32 GB or more available in Mac Pros and Xserves.

Apple is also advancing a new technology called OpenCL (for “open computing language”), which allows developers to spin off computationally heavy tasks into jobs that can be run on the fastest processors available to the system. The name is not accidentally related to OpenGL, which does a similar task for graphics operations to fully leverage the GPU in an abstracted way that works on any graphics card.

OpenCL unlocks the GPUs and the multiple cores in modern CPUs for developers who are writing code that deals with graphics and media processing or just heavy math and physics calculations (such as games). Using just slightly specialized code, OpenCL’s just in time compiler prepares developers’ tasks to run on the most appropriate processing engine available on the system, which is increasingly going to be the graphics card.

OpenCL also integrates with Grand Central to manage all those packetized tasks across multiple GPUs and the multiple cores of multiple CPUs in today’s systems. This will also open the door for Apple to include its own acceleration hardware technology in a way that will be easy if not automatic for developers to use.

More New Than the Who’s-Who Knew, Too.

Of course, there’s a lot more that all this going on in Snow Leopard. In fact, there’s so much that’s new that Apple’s line about “no new features” is a bit misleading. Snow Leopard pushes things ahead in a way that will confuse and befuddle tech pundits used to arranging Apple’s marketing names like refrigerator magnets. The next article will attempt to clear things up with a look at Myths of Snow Leopard.

Apple is marketing the idea of there being “no new features” for Snow Leopard and instead promising an overall improvement in how Mac OS X works under the hood, thanks to a diligent code optimization and refactoring cycle discussed in the previous article. At the same time, there are plenty of significant new features coming in Snow Leopard to look forward to. Here are ten big new features (plus a few minor ones) that you probably haven’t heard much about from anywhere else, including my previous articles on the subject that already described QuickTime X, Grand Central, and OpenCL.

Pulling Invisible New Features into Snow Leopard.

Apple’s increasing collaborations with the open source community have pulled back the veil of secrecy on several new but mostly invisible enhancements that will be showing up in Snow Leopard.

One relates to LLVM, the Low Level Virtual Machine compiler architecture project originally founded at the University of Illinois. Apple began contributing to LLVM development in 2005, and started using it Leopard to expand support for OpenGL hardware features. Lower-end Macs that lack the silicon to interpret that specialize graphics code can now do it in software.

LLVM is also working its way into Apple’s Xcode IDE, initially as a highly efficient optimizer and code generator that works as a bolt-on upgrade to components of GCC, but eventually as a complete compiler replacement. That project, known as Clang, was opened up last year. LLVM compiler technology not only makes developers more productive, but also results in code that runs significantly faster on the same hardware.

Apple’s other open secret: the LLVM Complier
The LLVM Compiler Infrastructure Project

Another openly hidden secret in Mac OS X is CUPS, the Common Unix Printing System. Beginning with Jaguar in 2002, Apple adopted and licensed CUPS from its developer as Mac OS X’s printing engine. It then purchased the project outright. CUPS is also the de facto printing system for Linux distros and is available for BSD and other commercial Unix systems.

That means Apple owns the project that develops the printing architecture for Linux. That’s not an issue because Apple has established a reputation in open source as a strong contributor and open sharer. According to a review of bug fixes and improvements in CUPS software, 24% of the enhancements came from Apple while 76% came from free and open source software contributors working with Linux, OpenSolaris, and other projects. Of course, 100% of both sides benefited from that sharing.

CUPS collaboration has resulted in high quality code and the advancement of new features. CUPS 1.4, the version sources say Snow Leopard will use, adds performance enhancements and a variety of security improvements that use sandboxing to prevent malware attacks on the printing system from being able to read sensitive documents that may be in use by printers.

Common UNIX Printing System

A third significant new feature originating from an open source project in Snow Leopard is ZFS support, portions of which come from the OpenSolaris project (along with Sun’s DTrace technology, which Apple uses in its Instruments performance profiling tool). Leopard debuted read-only ZFS features, but Snow Leopard and Snow Leopard Server will provide both read and write support for Sun’s new 128-bit file system. ZFS was designed to provide “simple administration, transactional semantics, end-to-end data integrity, and immense scalability.”

ZFS hype during the development of Leopard helped the new file system reach buzzword status as news of the three letter acronym swept through blogs and the tech media. It is frequently described as being the imminent replacement for the Mac’s native HFS+. However, the benefits of ZFS including as storage pooling, data redundancy, automatic error correction, dynamic volume expansion, and snapshots all apply primarily to servers and higher-end workstation users who deal with multiple disk drives.

ZFS isn’t going to replace HFS+ outright in Snow Leopard, and has limited relevance today to desktop and laptop users, particularly those who never move beyond the single disk drive installed in their system.

Pushing Visible New Features in Snow Leopard.

Apple’s extensive work in developing push support for Exchange Server on the iPhone will also be included in Snow Leopard’s Mail, Address Book, and iCal. Push support in those client side apps are also being used to power MobileMe’s push messaging subscription service and Snow Leopard Server’s push messaging services. Apple will be offering both in parallel as alternatives to Exchange, thanks to smart planning on the part of Apple’s engineers to develop an interoperable push architecture in Mac OS X and on the iPhone.

There is also a fourth application of push that has developed alongside push messaging: Apple’s new Push Notification Service. PNS allows iPhone and iPod touch users to set up server side notification alerts that don’t require mobile applications to stay running in the background just to update users of the external events they track. Along with Bonjour discovery, PNS will keep iPhones wirelessly connected in all sorts of sophisticated ways that third party developers can imagine in their applications.

Whether Apple will integrate a listener for the same PNS system into the desktop side of Mac OS X remains to be seen, but it would allow a single, unified interface for alerting client users of new events. I proposed a system wide, Growl-style notification system in the Leopard Wish List published back in 2005.

Snow Leopard Server Takes on Exchange, SharePoint
Apple’s Mobile Me Takes On Exchange, Mobile Mesh

With the strong push into push messaging, Apple will make mobile devices even more tightly integrated with its desktop products. Leopard delivered Back To My Mac as a novel way to use Wide Area Bonjour’s dynamic service registration as a mechanism for sharing resources served from home to any location without configuring static naming services for address lookups. Because any software can register itself with .Mac/MobileMe, this opens the door to third party developers with the vision to exploit the potential of these enabling technologies.

A Global Upgrade for Bonjour: AirPort, iPhone, Leopard, .Mac
Ten Big Predictions for Apple in 2008

Among the technologies profiled earlier in Myth 3 that have been trickling from the iPhone into Mac OS X, there’s at least one idea I proposed for the iPhone that will be in Snow Leopard’s Safari: self contained web apps. The new feature will allow users to run web applications as a local app in its own window, essentially making the web platform into a native-looking app that can run outside of Safari.

I proposed a similar feature as a possibility for the iPhone prior to the announcement of the Cocoa Touch SDK: web apps packaged up into a set of files that could be run on the device as a Dashboard widget-like standalone app, even when off the network. Why Apple hasn’t pursued such an obvious strategy is a little hard to figure out, but it seems they’ve got the ball rolling on the desktop.

That ball will be rolling even faster thanks to SquirrelFish, a new JavaScript interpreter that will make Safari and any other WebKit-based browsers, standalone self contained apps, and Dashboard widgets all a lot faster. Apple’s MobileMe, Yahoo’s Flickr, and Google various web apps will all gain new speed thanks to faster JavaScript execution. SquirrelFish will also raise the bar in performance and efficiency in the Rich Internet Applications sector in general, giving Flash, Silverlight, and Java a faster, simpler, and more openly interoperable runtime to compete against.

Microsoft’s Application Features in Mac OS X, System Wide.

Microsoft’s business model of tacking on features hasn’t been a total wash. The company’s desperate efforts to invent novel marketing features for every new release of Windows and Office have pioneered a number of ideas that have later found their way into Mac OS X. One example is the idea of Fast User Switching, which Apple added to Panther. Windows XP pioneered the trick, but built it upon the kluge that is Terminal Services.

Microsoft also helped originate the basis of Ajax web apps by inventing XMLHttpRequest in order to make its Outlook Web Access 2000 web app work decently within Internet Explorer. Today, standards-based web apps are eating a hole into Microsoft’s monopoly on the proprietary desktop platform, and tools such as SproutCore and resulting products such as MobileMe are poised to tear down interoperability barriers and level the playing field. Microsoft may now regret having opened Pandora’s Box in terms of standards-based web applications, but its efforts to seal the web back up with the proprietary Silverlight plugin, which turns web apps into .NET programs, will now be next to impossible.

Another example of a Microsoft innovation are the fancy text features in Word, such as red underlining to highlight spelling mistakes and the green squiggle for grammar errors. Word also features a variety of word auto correction, smart dash insertion, and text replacement features (such as typing TM to get the ™ character). The former have already become system-wide features in Mac OS X, while sources indicate that the latter text processing features will find their way into Snow Leopard, and therefore every application that runs on it.

Super Size Me.

On top of injecting Word features into its OS for the use of every application, Apple will also expand the use of its own Data Detectors, a technology it invented in the mid 90s for identifying useful bits of text and making it actionable. Leopard introduced Data Detectors in Mail as a way to extract contacts and events for use in Address Book and iCal, but Snow Leopard will expose Data Detectors everywhere it draws text.

Sources also indicate Snow Leopard will expand upon Font Book to provide full Auto Activation of any fonts requested by any application, using Spotlight to track them down. Snow Leopard is also suggested to have a new set of frameworks specifically for working with multitouch trackpad gestures, patterned after those introduced with the MacBook Air.

Speaking of the ultra-thin Air, sometimes less is more. However, the high cost and relatively low capacity of Solid State Drives like the $1000, 64 GB SSD option offered for the Air means that one Microsoft feature Snow Leopard could do without is bloat. As one reader noted, “Currently, Leopard requires 9 GB of available disk space for installation and iLife requires an additional 3 GB. This means that a product such as the [SSD] MacBook Air comes with the hard drive 20% full.”

How the MacBook Air stacks up against other ultra-light notebooks

Think Small.

Snow Leopard aims below the bloat to accommodate the coming wave of SSD-based systems. In the latest build, sources say Apple’s own apps are losing weigh dramatically across the board. The apps in the Utilities folder all drop from 468 MB to 111.6 MB, for example. Other apps are similarly svelte, as the graph below indicates.

snow leopard apps are losing weight

Is this the product of just code optimization and shared resources? One factor likely relates to work on Resolution Independence, which substitutes bitmapped raster graphics (which define every pixel) with smaller vector graphics files (which draw GUI elements and controls by recipe).

Vector graphics can be scaled to any size while retaining a high quality appearance, while bitmapped graphics can quickly look blocky when scaled up. Adding larger bitmapped versions can solve that problem, but at the cost of consuming more disk space. Apple earlier told developers it would be providing a library of shared, high quality vector graphics they could use instead of each packaging their own bitmapped art into every app.

The dramatic size reductions in these apps must also involve more efficient Localization. For example, Mac OS X Leopard’s Mail currently weighs in at over 285 MB, but the majority of its bulk comes from 18 language localizations inside the application bundle that consume 276 MB. The actual Universal Binary code is only a few megabytes and even its associated graphics and other resources only amount to 2.8 MB.

Why does Apple default to dumping support for 18 or more languages in every app without providing any simple, centralized way to get rid of the unnecessary ones? Perhaps that question is answered in Snow Leopard, where Mail is reportedly just 91 MB. That’s too big to simply to be an English-only, stripped down version for developers, but still far smaller than than Leopard’s. Across the board, it appears Snow Leopard apps are about a third as large as their Leopard equivalents.

And so while Snow Leopard paradoxically gains more useful features through code improvements and under-the-hood retooling rather than from a Microsoft-style new feature focus that aims to deliver “wow” with flashy marketing gimmicks, the system is also getting smaller and tighter. There must also be some other subtraction, right? Will Snow Leopard scrape away the old Carbon API? That’s the next myth.

No comments: