Wondering what you can do with a hacked Kinect on your Mac? First interesting example is this: A new processing app by Chris Rojas takes the distance data and renders objects in neon cubes, with the size of the cube based on the object’s distance from the sensor. And to make things a bit more interesting an iPad was integrated with the system for more control. An iOS app provided sliders to define and adjust different “planes of interest,” along with the accelerometer input of the iPad to control zoom and pan of the virtual camera.

Apple on Tuesday began selling a Magic Trackpad, a new wireless device that connects via Bluetooth to any Mac computer and allows users to manipulate what’s on their screens through gestures.

Like track pads on laptops, the battery-powered gadget can be used instead of a mouse or in conjunction with one. The pad’s entire surface is a button that clicks, so that people can use it in place of a mouse.

“Swiping through pages online feels just like flipping through pages in a book or magazine. And inertial scrolling makes moving up and down a page more natural than ever,” says Apple on its Magic Trackpad web page.

Some details about the device had already leaked, and its official launch had been expected since it was mentioned at Apple’s developers’ conference in early June.

The Magic Trackpad is made from glass-covered aluminum and is the same angle and height as a Mac desktop keyboard, which it’s designed to sit next to. It comes as a standalone accessory and costs $69.

Apple says the device supports a set of gestures that include two-finger scrolling, pinching to zoom, rotating with your fingertips, three-finger swiping, or switching between applications with four fingers.

“Go from typing to gesturing in one motion, or do both at the same time,” Apple says in its product description.

Apple began selling the Magic Trackpad in its stores and on its website Tuesday morning along with an updated line of iMac and Mac Pro computers. Its online store was down for a period of time beforehand, causing the usual fevered web speculation about what the closely watched company might introduce.

Magic Trackpad quickly became a trending topic on Google and Twitter, although not everyone was impressed.

But quietly, the Slate went away, and now the buzz around HP is that it will use Palm’s WebOS as the foundation for iPad rivals, once it’s completed its buyout of Palm. (On Friday, Digitimes quoted an HP Taiwan exec saying the Slate would use WebOS instead of Windows 7.)

All the tablet buzz now centers around the iPad, various Android devices said to be in development at Dell and other manufacturers, and HP’s future WebOS tablet. What happened to Windows 7?

A Tablet Is Not a Laptop Whose Screen Is Always Visible
The answer: The iPad proved a tablet shouldn’t be a portable computer that happened to have its screen always exposed. Instead, a tablet should be something else. Apple got a lot of criticism early on for not making the iPad essentially a Mac OS X tablet computer, in the vein of the Windows tablet computers available — but hardly used — for the last decade.

Apple — followed by Dell, HP, and the rest of the industry — has realized a tablet is something different, and force-fitting a desktop OS into it simply won’t work. Remember the splash Microsoft and HP made on touchscreen PCs last fall? That chatter has gone quiet too outside the nichy kiosk space, and for the same reason: Windows 7 is not designed for a touch-oriented interaction. Microsoft’s touch extensions to Windows 7 are awkward to use and don’t get around the problem that all the apps and the OS itself assumes the use of mouse or other pointing device. A finger isn’t as accurate as a mouse, and UI elements designed for a mouse-and-keyboard interface don’t translate to the touch world, even with UI extensions that support finger-based input.

Lessons from Apple’s Touch-Native Enforcement
Microsoft needs a UI designed for touch — rich gestures for input and a fundamental UI design that doesn’t involve lots of elements such as tabbed panes, radio buttons, check boxes, and dialog boxes. But it doesn’t have one. Plus, for applications to really support touch and gestures, they need to do more than map mouse actions to finger ones; the interface and operational design needs to be touch-native as well. No mapping layer for libraries will take care of that for you, as you can quickly see if you use a Windows 7 touchscreen PC.

I believe Microsoft recognizes that fact, which is why its forthcoming Windows Phone 7 mobile platform uses a separate, largely new OS designed at the ground level for gestures and touch.

Could Microsoft retrofit Windows 7 to support touch natively through and through, making it appropriate for a tablet? Maybe. After all, the iPhone OS is based on Apple’s Mac OS X, a desktop operating system that supports the same UI expectations and complexity as Windows 7. A lot of the underlying code is the same between the Mac OS and the iPhone OS.

Yet you can’t run Mac OS apps on an iPhone or vice versa. Sure, some UI elements are the same across the two operating systems, but they have more to do with a consistent Apple style than with fundamental operations. Look no further than Apple’s iWork productivity suite for Mac OS X and iPhone OS: Beyond a compatible file format and name, they share little in common in terms of how they actually operate. (I’d argue that iWork for iPhone OS is a disappointment and harder to use than it should be, though that’s due to a murky interface, not to a usage of the desktop paradigm.)

The bottom line is that even though technical components are shared between the Mac OS and the iPhone OS, the irrelevant Mac OS functions aren’t gumming up the iPhone OS, and Apple’s development environment doesn’t let you pull through desktop approaches into your mobile applications. You’re forced to go touch-native.

Apple gets some advantage in both its internal development and its ability to cross-train Mac OS and iPhone OS developers by having that common core, even though the UI and app results are very different. Theoretically, Microsoft could do the same with Windows 7 and a tablet version of Windows Phone 7 by giving them a common core that doesn’t impose itself on the user in ill-fitting ways, as is the case with Windows 7′s touch extensions.

So far, Microsoft has chosen not to do so; instead, it is keeping the desktop and mobile OSes separate. It did the same with Windows 2000/XP/Vista/7 and Windows CE/Mobile, but foolishly imposed the desktop UI onto the separate mobile OS, so its developers ended up applying the same application approaches to two separate operating systems, creating the UI disconnect in the mobile environment. HTC and others tried to mask that with UI overlays, but you quickly found your way back to the desktop Windows interface as you used applications.

By comparison, Apple rightfully had its developers focus on creating different types of native applications for two related operating systems, with user-pleasing results.

Microsoft seems to have switched to Apple’s strategy
Microsoft seems to have learned that lesson by providing developers tools for Windows Phone 7 that work in its .Net and XNA development environments, which are familiar to desktop developers. That’s smart, as it leverages what Microsoft developers know but doesn’t impose desktop assumptions on the mobile environment — with a likely result similar to what Apple achieved for its Mac OS X/iPhone OS developers.

But Windows Phone 7 is about smartphones, not about tablets. It’s not at all clear if Microsoft still harbors hopes of Windows 7-based tablets — as Ballmer did so publicly in January — or if it will change gears and evolve the Windows Phone 7 OS to support tablets. In other words, will it stop trying to force-fit Windows onto tablets and adopt Apple’s approach of evolving a mobile OS for tablets? Google and Palm/HP are taking Apple’s approach for tablets, evolving Android OS and WebOS for tablets.

I’m betting that Microsoft will do the same thing.

Sure, a few Windows 7 slate-style tablets will ship — Asus and MSI are said to have models shipping later this year. But those products will go nowhere, because Windows 7 is simply not the right operating system for a slate. (These companies also made a lot of noise around Android slates at the CES show in January, but now seem to have cooled to the idea, a reflection of their short-term market strategies.) And that’s why you won’t see Windows 7 Slate or WinPad or whatever outside tech blog photos.

| Location-based operating system
| Multiple widget desktops designed around a location or activity ie Kitchen, Office, Car
| Automatically switches between desktops with GPS and wi-fi mapping
| Simplified Collections menu allows browsing via function rather than application

Though handset makers buy their touchscreens as components from the same select pool of suppliers, a good touchscreen experience requires more than just hardware. It requires a bit of design alchemy blending software, engineering and calibration for the perfect feel. Few smartphone makers have managed to get that balance right, say experts.

“If you think that no other touchscreen out there is as good as the iPhone, its not all in your head,” says Chris Verplaetse, vice president of the Moto Development Group, a product design and development firm. “It’s like asking what makes a Mercedes door close like a Mercedes door and a Hyundai door close like one though they use the same steel. There’s clearly a difference.”

Variables include engineering details such the calibration of the touch sensor so it can separate the signal from the noise, the quality of the firmware and the level of integration of the touch experience into the phone’s user interface. There are also more difficult-to-quantify things such as as the level of the company’s commitment to making the best touchscreen experience possible.

“Many layers account for the performance of a touchscreen,” says Verplaetse. “But it all comes down to how well the electronics and the mechanical hardware are integrated.”

As cellphones became more powerful, allowing users to surf the internet and check e-mail, handset makers started to add touch capability to their phones. The earliest screens were resistive touchscreens, where two thin metallic layers are separated by a narrow gap. A finger pushing down on the top layer makes contact with the bottom surface and the point of contact is computed by the accompanying electronics.

But resistive touchscreens didn’t make most consumers happy because they weren’t responsive enough — you had to really push and hammer away at the display with your fingernail or a stylus to get it to respond.

The capacitive touchscreen in Apple’s iPhone changed the game, because it’s not pressure-sensitive. Instead, this kind of technology responds to the electrical properties of your skin, not the pressure of your finger, to figure out where you’re touching the screen. For the first time, just a light tap could open an application or a flicking gesture could get the screen scrolling. Best of all, it seemed effortless.

A projected capacitive touchscreen — the kind that’s usually used in phones — has a glass insulator coated with a transparent conductive layer. The layer is etched into a gridlike pattern. When a finger touches the surface of the screen, it distorts the electrostatic field. That can be measured as a change in capacitance. The location of the touch is computed and it is passed on to a software application that relates the touch into actions for the device.

In theory, all capacitive touchscreens should offer consumers the same experience, but they rarely do, says Andrew Hsu, a technology strategist for Synaptics, one of the biggest touchscreen component makers.

“Capacitive touch-based handsets involve a lot of development work and quite a bit of engineering expertise in order to give them their ‘magical’ quality,” says Hsu.

It’s Not Just About Hardware

Smartphone users have no way to measure exactly how well the capacitive sensor system on their phone is actually working. Their perception is based on the feedback they see on the screen, says Hsu. That means a touchscreen could be quite fast and accurate, but if the visual display doesn’t keep up, it won’t feel smooth or responsive.

That’s where well-designed user interfaces and quality firmware come into play.

“Some systems are better at it than others,” says Hsu.

Synaptics ran tests comparing the iPhone touchscreen to the original BlackBerry Storm. They found that the Storm’s touchscreen sensor responded well, which pointed the finger at the underlying firmware.

It’s also a reason why BlackBerry maker Research In Motion was able to fix some of the lag and the bugginess of the screen that reviewers had initially complained about. Subsequent updates to the Storm’s software significantly improved its responsiveness to touch.

Another problem is separating signal from noise, which some phones are better at than others.

A perfectly designed and well-tuned capacitive sensing system would require no pressure to detect the presence of a user’s finger. But to get there, handset makers have to solve what Hsu calls the “needle in a haystack problem.”

The amount of signal that your finger contributes when it touches the sensor is very small compared to the noise already present in the system. To accurately sense it and compute its location requires some software magic.

“Even if you design the entire touchscreen right, once you put it into the device, there’s an impact from other sources that emit electromagnetic interference, such as the wireless unit,” says Hsu.

That’s where an ASIC, or application specific integrated circuit, is needed to measure and amplify the signals. Apple reportedly designed its own ASIC for the iPhone’s touchscreen, while most other companies buy an ASIC from one of the touchscreen chipmakers.

Finding the Right Balance

In January, Moto Labs, Moto Development Group’s research group, tested different touchscreens by using a drawing program to draw a few straight lines on the display. On a good touchscreen, users can draw clean, straight lines, say the researchers, while inferior touchscreens show lines that look jagged (see the photo above). The jagged lines happen because the sensor size may be too big, the touch-sampling rate may be too low, or the algorithms that convert gestures into images don’t faithfully represent user inputs.

Moto Labs’ tests showed the iPhone had the most precise lines, though there was loss of sensitivity around the edges. The HTC Droid and Nexus One also did well, while the Motorola Droid’s touchscreen came out at the bottom.

Though Palm Pre wasn’t included in that test, Moto says the Pre’s screen responded well. Meanwhile, a Palm Pre Plus user recently complained about the device’s poor touchscreen.

Palm wouldn’t comment about that particular complaint, but the company says the Pre and Pre Plus have been designed to optimize the touch experience. For instance, the Pre and the Pre Plus have a “soft arc” on the top surface to allow for gestures without interference from the edges. And Palm tries to tweak the touchscreen through firmware updates.

“We made touch a central part of the navigation on the device,” says Leslie Letts, a spokesperson for Palm.

Getting the perfect touchscreen experience is also a matter of finding the right balance for the display.

One reason why Apple’s touch sensor is so sensitive to light touch is that the company uses a 12-volt power source for the sensing lines in the touchscreen sensor, versus the 3- to 5-volt power source that most other component manufacturers have. That higher voltage drive takes a toll on the battery life because it uses up more power, but it also translates into more accurate sensing, which means a better touch experience, say researchers at Moto.

Other cellphone manufacturers are making the investments to catch up but it will be a while before all touchscreens work as well as Apple’s, says J. Daniel Hebert, CEO of Moto Development Group.

“There’s a big difference between deferring to a vendor and driving the performance of the system,” he says.

One final factor: Steve Jobs. His imperious management style may be off-putting, but he does exercise complete control over the final versions of his company’s products, and that hands-on approach at the company’s highest levels helps ensure that the whole experience works well — including the feel of the touchscreen.

“For a good touchscreen, you need someone who is a generalist who can ask more than whether the software is complete and whether the hardware is complete,” says Verplaetse. “Steve Jobs is an example of someone who probably asks, “Does this feel right?” when he’s looking at a new product.”

With a multitouch mouse, a user can, for example, browse through a virtual stack of digital photos by flicking a finger across the mouse’s surface, rotate an image by stroking the mouse, or zoom in on a picture by drawing an arrowhead with a fingertip.

“If the [traditional] mouse pointer is your virtual fingertip, we’re giving you a virtual hand,” says Dan Rosenfeld, a researcher with Microsoft’s Applied Sciences Group in Redmond, WA. There are multitouch surfaces for tabletops, computer monitors, and cellphone screens, he says, but aside from Apple’s new device, “there’s really nothing addressing the kind of tasks that lots of people do all day long, sitting in front of a desk at a computer.”

The first mouse outlined in the Microsoft research paper consists of a piece of clear acrylic lit with infrared light along its edge, where it attaches to a palm rest. Fingertips on the acrylic scatter the light, and an infrared camera captures the light patterns to track the movement of the fingers. The technique, known as frustrated total internal reflection (FTIR), has been used for other multitouch systems before, but this is the first design that also integrates the classic features of a mouse such as an optical sensor underneath and clickable buttons.

Another prototype, the dome-shaped Orb Mouse, also uses an infrared camera and light, but it reflects the light out of its center to make its entire hemisphere touch-sensitive. The dome also acts as a giant click button.

SideMouse, in contrast, positions the palm of the user’s hand on top and projects infrared light out of its side to track the user’s fingers as they move along the table next to the mouse.

The Cap Mouse abandons the infrared scheme altogether, instead tracking finger movements with a grid of capacitive sensors on its surface. Unlike the mice that rely on infrared technology, Cap Mouse isn’t affected by ambient lighting, consumes less power, and offers a less detailed account of finger movements.

Finally, the Articulated Mouse, also known as Arty, features two mini-mouse finger rests connected to the base. Each of the three parts contains an optical sensor for tracking movement, so that Arty can be manipulated by moving the base as well as each of the mini-mice separately.

In each of the five prototypes, although the user still moves the cursor across the screen by moving the entire mouse across the desktop, multitouch functions are accessed by moving individual fingers. Software created by Microsoft lets the user control their computer using these multitouch functions.

Shahram Izadi, a TR35 award winner who worked on Microsoft’s multitouch mice, says there’s still much work to be done on all of the prototypes. In particular, he says, the researchers need to determine the most natural way for users to switch between multitouch capabilities and standard mousing action. Activating the multitouch features with an extra mouse click makes the device slightly more difficult to use, but having those features available all the time means they might be accidentally triggered. “Users wanted to click the device to trigger the multitouch,” Izadi says. “But it’s not the most ergonomic form to click and then gesture.”

Alan Hedge, a professor of ergonomics at Cornell University who has worked on multitouch devices in the past, says multitouch can be useful, but he isn’t sure it’s right for computer mice. “The real benefit of multitouch is when you can take the whole top of a table or desk and use that to drag things around,” he says. “Confining it to the size of a mouse might actually slow you down,” particularly because clicking is so efficient.

Rosenfeld says that while efficiency is important, it’s not the only goal. A good tool, he says, should be both efficient and delightful. Ideally, a device lets you focus on your task and gets the job done, “but it’s also just a really lovely object to use.”