touch

A Touchy Subject

by Joey deVilla on August 1, 2010

win phone latitude xt2 surface

I think that we – and by we, I mean we developers and developer evangelist types at Microsoft – get touch and tablets, or slates, or pads, or whatever you’d like to call them, better than the Ars Technica article Ballmer (and Microsoft) still doesn’t get the iPad (written by Peter Bright and posted in the One Microsoft Way section) implies. I believe that over the next few months, you’ll see some interesting touch-related stuff coming from Microsoft, and that we have a responsibility to help developers understand the differences between mouse/keyboard computing and touch computing.

In anticipation of this, I’ve been make my move towards touch- (and other sensor-based) computing over the past little while, by migrating to the following devices:

The idea behind this purposeful move towards touch-equipped devices is to truly understand touch-based interfaces, which UI elements work and which ones don’t, and then to pass the lessons learned to my audience – developers and designers, whether you build for the Microsoft platform or the platforms of the Esteemed Competition.

My own move towards touch-based devices is a microcosmic example of the larger changes taking place at The Empire. The move to touch interfaces is taking place on Microsoft computing platforms of all sizes:

windows phone

As the Ars Technica article points out, one of the signs that we do get touch is the new interface design of Windows Phone 7. The design philosophy is build around touch (and other sensors), and the WP7 “design bible”, the Windows Phone User Interface Design and Interaction Guide [12 MB PDF], explains this philosophy beyond the mere technical details. Here’s the introduction to its section WP7’s touch interface (any emphasis in the quote below is mine):

Touch input is a core experience of Windows Phone 7 and has inherent differences from traditional keyboard and mouse input systems. Designed for natural and intuitive user interaction, touch input in Windows Phone 7 enables users to interact with application content such as a photo or a web page. Touch input enables simple and consistent user touch gestures that imitate real life behavior, such as panning on a photo to move it. Single-touch gestures make interaction easier with one hand, but multi-touch gestures are also available to provide more advanced gesture functionality.

Application developers should strive to create unique and exciting experiences that encourage the discovery of content through the use of touch gestures. Users should enjoy the experience of navigating through the steps of a task as well as the completion of the task itself. Touch gestures should provide a delightful, more colorful, intuitive experience within applications

Touch delights the senses as the user gets to see the interaction match the performance. The touch UI should always have aware and responsive performance, just like how real world objects respond to touch immediately, and applications on Windows Phone 7 should as well, by performing the action in real time and by providing immediate feedback that an event or process is occurring. Users should not have to wait as it breaks their immersion, flow, and concentration, especially as their gestures transition from one to the other. For example, a pan may turn into a flick or a tap can become a double tap, and the user should not be aware that the UI is switching gesture support.

There’s a great amount of understanding behind the nuances of touch-based interfaces in the Windows Phone User Interface Design and Interaction Guide, and over the next few months, we’ll be covering them in great detail in this blog.

big-ass table

When the Surface, a.k.a. the “Big-Ass Table”, came out, a number of people asked why such a big, expensive thing was built and what practical purpose such a beast would serve.

For starters, there are a number of customers who use it, from casinos in Vegas to bible study classes in megacurches to places closer to home (by which I mean Canada), from the company that did the security for President Obama’s visit to Ottawa to super-sexy Toronto design firm Teehan+Lax to Ontario College of Art and Design to Infusion, who’ve built applications such as Noront Resources’s GSI Surface tool to the security app Falcon Eye.

Equally important are lessons to be learned about input from touch and other sensors from a “concept” machine like the Surface, whose built-in camera systems allow for way more touch points than a resistive or capacitive touch screen will allow, as well as the ability to “see” objects on the tabletop. By being empirical and building such a computer, developing software for it and watching people interact with it, we learn more about touch and sensor-based computing way more than we could from mere theorizing.

I think Des Traynor captured our intent quite nicely in his article about Surface and other Microsoft efforts in the field of user interface:

When the Surface was released two years ago it was chastised by the public. The joke at the time was: “Apple and Microsoft both invest in multi-touch technology, Apple release the iPhone, Microsoft release a $15,000 coffee table!”.

But Surface wasn’t about “re-inventing the coffee table”, so much as it was prototyping a vision of the future of computing. There will come a time when “gathering around a laptop” will seem as ridiculous as connecting an ethernet cable; a time when everyone gathers around a multi-user computer to have a meeting or debate a design. With something like surface, Microsoft are preparing for that day.

standard computers

A lot of the knowledge from Surface applications have been injected into Windows 7 in the form of the Windows 7 Touch Pack. This pack gives Windows 7 a touch-based API and a set of apps originally designed for the Surface, so that they can run on touch-enabled computers, such as HP’s TouchSmart series, touch-enabled laptops like my own Dell Latitude XT2 as well as any computer connected to one of the new touch-enabled monitors (our manager John Oxley has one in his office).

The Ars Technica article goes on and on about Windows 7’s standard interface controls being too tiny for touch, but a quick look at the Touch Pack apps reveals that they don’t use the standard controls; rather, they use controls better-suited to touch. Here’s a screenshot of Surface Collage, the photo-collage application, running on my XT2:

surface collage

No standard Windows controls here! You manipulate the photos directly using gestures, and the strip along the bottom is a photo list, which you also manipulate through gestures. The closest thing to a standard Windows control is the “close” button near the upper-right hand corner of the screen, which is larger than the typical “close” button – small enough to be out of the way, yet large enough to click with a finger.

Here’s another app from the Touch Pack, Surface Globe, also running on my XT2:

surface globe

Once again, no standard Windows 7 controls here, but a map that you directly manipulate, augmented by finger-friendly controls.

The Touch Pack apps all follow this philosophy: when going touch, eschew the standard Windows 7 UI controls in favour of touch-friendly ones, and then back to bog-standard Windows 7 when exiting them. These apps show not just that we understand that touch computing is a different beast from mouse-and-keyboard computing, but that we also understand where they intersect.

future
We’re working on what I like to call “the touch continuum”, which spans pocket devices such as the Zune HD and Windows Phone, to portable computing with netbooks, laptops and soon, tablets, to desktop and tabletop and wall-sized units. And yes, we get that new types of user input call for new user interfaces and give rise to new usage patterns. We’re aware of the challenges of touch (and other sensor) input and over the next little while, you’ll see our answers to those challenges. And better still, we’ll share what we’ve learned in order to make you better developers and designers of software that use these new interfaces.

This article also appears in Canadian Developer Connection.

{ 1 comment }

Windows Phone 7 Series: Now That’s More Like It!

by Joey deVilla on February 15, 2010

Windows Phone 7 Series generic phone

A New Windows for the Phone

Ever since joining The Empire, I’ve been saying that Windows Mobile needs to go back to the drawing board. While there was good technology lying in its innards – mobile versions of the .NET framework, SQL Server and Office – treating the mobile form factor as “the desktop, but much, much smaller”, was the wrong approach. In the meantime, the Esteemed Competition were doing the right thing: designing their phones’ OS features and interface from the ground up rather than attempting to force-fit the desktop UI into a pocket UI.

Today at Mobile World Congress in Barcelona, Microsoft previewed the latest in a series of steps forward – consider Xbox to Xbox 360, Windows Vista to Windows 7, Live Search to Bing – there’s now Windows Phone 7 Series.

(The name’s a bit long. Whoever does the naming at Microsoft corporate HQ must get paid by the syllable.)

A Quick Look at Windows Phone’s Experience

A good starting point is this video, which covers Windows Phone’s features in three minutes, thirty seconds:

You can take an interactive tour of the UI at the Windows Phone 7 Series site:

Screenshot of the Windows Phone 7 Series site's home page

A Closer Look at the Windows Phone Experience

Over at Channel 9, Laura Foy has posted her interview with Joe Belfiore, VP Windows Phone 7 Program Management, who gave her a walkthrough of the goodies in Windows Phone (the video is 22 minutes, 18 seconds):

Get Microsoft Silverlight

Some quick notes from the video:

  • There are three mandatory hardware buttons, which are context-sensitive:
    • Back
    • Windows (the “Start” button)
    • Search
  • The screen is a capacitive touch-screen, capable of supporting multi-touch
  • The Start menu is built up of tiles: little block representing the information and features that you care most about
    • You can add your own custom tiles; Joe shows a “me” tile linked to his Facebook profile
  • A browser with:
    • Snappy performance
    • Support for multitouch actions such as pinch zoom, double-tap to zoom and finger drag
    • Very readable text, that to sub-pixel positioning in HTML
    • Phone number recognition in HTML documents; touch them to dial them
    • Street address recognition in HTML documents; touch them to get a map
    • Multiple tabs
  • The “People Hub”
    • Aggregates Exchange, Hotmail, Gmail, Yahoo! Mail and other mail contacts
    • Provides a live feed of your contacts
  • Context-sensitive search:
    • Press the “Search” button while in the People Hub, and you search your people list
    • Press the “Search” button while in the Start menu, and it runs a web search
      • Based on your query, it knows whether to give you a web search result or a local search result
      • In the demo, Joe does a search for pizza and gets a map and results for pizzerias near him, and a quick pan over to adjacent pages yield directions and reviews
      • A tap on “nearby” yield the locations of useful things like parking, ATMs and so on near the selected pizzeria
      • In another demo search, Joe does a search for “Avatar” and it returns a list of nearby theatres and times for the movie Avatar; a quick pan to an adjacent page yields the results for local business and places with “Avatar” in the name
  • Email:
    • Easy pivoting between unread, flagged and urgent emails
    • A caching system prevents you from seeing the dreaded “loading” screen
    • Press “Search” within email and you perform a search of your email messages, by subject, text and so on
  • Rotation: you can operate the phone in “portrait” or “landscape” mode
  • Calendar:
    • Support for both work and personal calendars
  • ActiveSync works in the background and keeps the phone synced with email, contacts and calendar
  • User-customizable UI colour schemes
  • The “Pictures Hub”
    • Gallery: Lets you browse all the pictures on your phone
    • Mosaic: Recent and favourite pictures
    • What’s New: New photos from your social networks
    • Camera roll: A folder for photos taken with your phone
    • Support for photo albums from Facebook and Windows Live, which you browse as if they lived right on your phone
  • Music and Video
    • History: Most recently played music and videos
    • New: New music and videos added since the last sync
    • Zune HD-style marketplace searching and support for Zune subscriptions with unlimited music plays
  • The “Me” tile
    • Lets you update your status on places like Facebook
    • Nice little typing features like auto-spelling-correction and a special soft keyboard for emoticons
  • The UI concept: Windows Phone is task-centric, not app-centric, with a hub associated with each: people, photos, media
  • There’s also a games hub, which ties into Xbox Live
  • Third-party applications and games? Wait…

Wait a Minute…What About Third-Party Apps and Games?

"MIX10: The Next Web Now" logo buttonCan you wait a month?

Here’s the deal: the announcement at Mobile World Congress was about showing what Windows Phone can do. As for what’s possible on the developer front, it’ll all be announced at the MIX10 Conference, which takes place from March 15th through 17th in Las Vegas.

There will be a dozen sessions at MIX10 for Windows Phone, and they promise to be quite interesting. I’ll be at MIX10, and will blog what I learn from these sessions when they take place.

You can save $200 off the price of MIX10 registration if you register before February 21st, so if you want to get in on the ground floor with Windows Phone and save some money, register now!

What the Tech Press is Saying

Pretty good stuff, actually. Rather than bury you with links to a zillion blog entries filed from Mobile World Congress, I thought I’d pick two of the big tech blogs, Gizmodo and Engadget:

Here’s what Gizmodo has to say about the new Windows Phone:

It’s different. The face of Windows Phone 7 is not a rectangular grid of thumbnail-sized glossy-looking icons, arranged in a pattern of 4×4 or so, like basically every other phone. No, instead, an oversized set of bright, superflat squares fill the screen. The pop of the primary colors and exaggerated flatness produces a kind of cutting-edge crispness that feels both incredibly modern and playful. Text is big, and beautiful. The result is a feat no phone has performed before: Making the iPhone’s interface feel staid.

If you want to know what it feels like, the Zune HD provides a taste: Interface elements that run off the screen; beautiful, oversized text and graphics; flipping, panning, scrolling, zooming from screen to screen; broken hearts. Some people might think it’s gratuitous, but I think it feels natural and just…fun. There’s an incredible sense of joie de vivre that’s just not in any other phone. It makes you wish that this was aesthetic direction all of Microsoft was going in.

Here are Engadget’s impressions, after having some hands-on time with Windows Phone:

The design and layout of 7 Series’ UI (internally called Metro) is really quite original, utilizing what one of the designers (Albert Shum, formerly of Nike) calls an "authentically digital" and "chromeless" experience. What does that mean? Well we can tell you what it doesn’t mean — no shaded icons, no faux 3D or drop shadows, no busy backgrounds (no backgrounds at all), and very little visual flair besides clean typography and transition animations. The whole look is strangely reminiscent of a terminal display (maybe Microsoft is recalling its DOS roots here) — almost Tron-like in its primary color simplicity. To us, it’s rather exciting. This OS looks nothing like anything else on the market, and we think that’s to its advantage. Admittedly, we could stand for a little more information available within single views, and we have yet to see how the phone will handle things like notifications, but the design of the interface is definitely in a class of its own.

(In another article, Engadget simply summed it up with “Microsoft is playing to win”.)

Watch this Space!

"Counting Down to Seven" badgeWe’ll have more announcements about Windows Phone over the next few weeks, so keep an eye on this blog!

This article also appears in Canadian Developer Connection.

{ 4 comments }