user interface

intro slide[3]

One of the tricky things about helping developers build for a platform that has yet to be released is that it’s a tabula rasa. There’s no history, which is both blessing and curse: we developers get to make that history, but at the same time, we’re working in the dark. There are no examples to emulate and no best practices to follow – it’s just us and whatever user interface guidelines there happen to be (which, in the case of Windows Phone 7, is the Windows Phone UI Design and Interaction Guide).

That’s why I’m glad that Microsoft is building WP7 apps like USGA Shot Tracker, a gorgeous golf scorekeeping app that practically announces to developers: “This is how you do it. This is how you write a usable, beautiful, truly Windows Phone 7 app.” Here’s a video of USGA Shot Tracker in action:

Give the app a look, and also make sure you check out the article on Long Zheng’s blog, istartedsomething, which includes images of USGA Shot Tracker’s screens.

Keep an eye on this blog, because I’m a couple of days away from starting an ongoing series on well-designed WP7 apps and how you implement them. I’ll take a closer look at USGA Shot Tracker and other apps, going through them with a fine-toothed comb in attempt to learn as much as possible from them, and share that knowledge with you.

This article also appears in Canadian Developer Connection.

{ 0 comments }

A Touchy Subject

by Joey deVilla on August 1, 2010

win phone latitude xt2 surface

I think that we – and by we, I mean we developers and developer evangelist types at Microsoft – get touch and tablets, or slates, or pads, or whatever you’d like to call them, better than the Ars Technica article Ballmer (and Microsoft) still doesn’t get the iPad (written by Peter Bright and posted in the One Microsoft Way section) implies. I believe that over the next few months, you’ll see some interesting touch-related stuff coming from Microsoft, and that we have a responsibility to help developers understand the differences between mouse/keyboard computing and touch computing.

In anticipation of this, I’ve been make my move towards touch- (and other sensor-based) computing over the past little while, by migrating to the following devices:

The idea behind this purposeful move towards touch-equipped devices is to truly understand touch-based interfaces, which UI elements work and which ones don’t, and then to pass the lessons learned to my audience – developers and designers, whether you build for the Microsoft platform or the platforms of the Esteemed Competition.

My own move towards touch-based devices is a microcosmic example of the larger changes taking place at The Empire. The move to touch interfaces is taking place on Microsoft computing platforms of all sizes:

windows phone

As the Ars Technica article points out, one of the signs that we do get touch is the new interface design of Windows Phone 7. The design philosophy is build around touch (and other sensors), and the WP7 “design bible”, the Windows Phone User Interface Design and Interaction Guide [12 MB PDF], explains this philosophy beyond the mere technical details. Here’s the introduction to its section WP7’s touch interface (any emphasis in the quote below is mine):

Touch input is a core experience of Windows Phone 7 and has inherent differences from traditional keyboard and mouse input systems. Designed for natural and intuitive user interaction, touch input in Windows Phone 7 enables users to interact with application content such as a photo or a web page. Touch input enables simple and consistent user touch gestures that imitate real life behavior, such as panning on a photo to move it. Single-touch gestures make interaction easier with one hand, but multi-touch gestures are also available to provide more advanced gesture functionality.

Application developers should strive to create unique and exciting experiences that encourage the discovery of content through the use of touch gestures. Users should enjoy the experience of navigating through the steps of a task as well as the completion of the task itself. Touch gestures should provide a delightful, more colorful, intuitive experience within applications

Touch delights the senses as the user gets to see the interaction match the performance. The touch UI should always have aware and responsive performance, just like how real world objects respond to touch immediately, and applications on Windows Phone 7 should as well, by performing the action in real time and by providing immediate feedback that an event or process is occurring. Users should not have to wait as it breaks their immersion, flow, and concentration, especially as their gestures transition from one to the other. For example, a pan may turn into a flick or a tap can become a double tap, and the user should not be aware that the UI is switching gesture support.

There’s a great amount of understanding behind the nuances of touch-based interfaces in the Windows Phone User Interface Design and Interaction Guide, and over the next few months, we’ll be covering them in great detail in this blog.

big-ass table

When the Surface, a.k.a. the “Big-Ass Table”, came out, a number of people asked why such a big, expensive thing was built and what practical purpose such a beast would serve.

For starters, there are a number of customers who use it, from casinos in Vegas to bible study classes in megacurches to places closer to home (by which I mean Canada), from the company that did the security for President Obama’s visit to Ottawa to super-sexy Toronto design firm Teehan+Lax to Ontario College of Art and Design to Infusion, who’ve built applications such as Noront Resources’s GSI Surface tool to the security app Falcon Eye.

Equally important are lessons to be learned about input from touch and other sensors from a “concept” machine like the Surface, whose built-in camera systems allow for way more touch points than a resistive or capacitive touch screen will allow, as well as the ability to “see” objects on the tabletop. By being empirical and building such a computer, developing software for it and watching people interact with it, we learn more about touch and sensor-based computing way more than we could from mere theorizing.

I think Des Traynor captured our intent quite nicely in his article about Surface and other Microsoft efforts in the field of user interface:

When the Surface was released two years ago it was chastised by the public. The joke at the time was: “Apple and Microsoft both invest in multi-touch technology, Apple release the iPhone, Microsoft release a $15,000 coffee table!”.

But Surface wasn’t about “re-inventing the coffee table”, so much as it was prototyping a vision of the future of computing. There will come a time when “gathering around a laptop” will seem as ridiculous as connecting an ethernet cable; a time when everyone gathers around a multi-user computer to have a meeting or debate a design. With something like surface, Microsoft are preparing for that day.

standard computers

A lot of the knowledge from Surface applications have been injected into Windows 7 in the form of the Windows 7 Touch Pack. This pack gives Windows 7 a touch-based API and a set of apps originally designed for the Surface, so that they can run on touch-enabled computers, such as HP’s TouchSmart series, touch-enabled laptops like my own Dell Latitude XT2 as well as any computer connected to one of the new touch-enabled monitors (our manager John Oxley has one in his office).

The Ars Technica article goes on and on about Windows 7’s standard interface controls being too tiny for touch, but a quick look at the Touch Pack apps reveals that they don’t use the standard controls; rather, they use controls better-suited to touch. Here’s a screenshot of Surface Collage, the photo-collage application, running on my XT2:

surface collage

No standard Windows controls here! You manipulate the photos directly using gestures, and the strip along the bottom is a photo list, which you also manipulate through gestures. The closest thing to a standard Windows control is the “close” button near the upper-right hand corner of the screen, which is larger than the typical “close” button – small enough to be out of the way, yet large enough to click with a finger.

Here’s another app from the Touch Pack, Surface Globe, also running on my XT2:

surface globe

Once again, no standard Windows 7 controls here, but a map that you directly manipulate, augmented by finger-friendly controls.

The Touch Pack apps all follow this philosophy: when going touch, eschew the standard Windows 7 UI controls in favour of touch-friendly ones, and then back to bog-standard Windows 7 when exiting them. These apps show not just that we understand that touch computing is a different beast from mouse-and-keyboard computing, but that we also understand where they intersect.

future
We’re working on what I like to call “the touch continuum”, which spans pocket devices such as the Zune HD and Windows Phone, to portable computing with netbooks, laptops and soon, tablets, to desktop and tabletop and wall-sized units. And yes, we get that new types of user input call for new user interfaces and give rise to new usage patterns. We’re aware of the challenges of touch (and other sensor) input and over the next little while, you’ll see our answers to those challenges. And better still, we’ll share what we’ve learned in order to make you better developers and designers of software that use these new interfaces.

This article also appears in Canadian Developer Connection.

{ 1 comment }

If You Speak Database, Science Needs Your Brain!

by Joey deVilla on May 4, 2010

 database brain

If you’re in the Greater Toronto Area, have basic knowledge of database queries and want to help a grad student with a research project, Zuzel Vera Pacheco, one of Greg Wilson’s students at University of Toronto, needs to borrow your brain! In exchange, you’ll get a chance to win a $100 Best Buy gift card.

Here’s her description of the project:

Want to win a $100 Best Buy gift card? Do you have basic knowledge about database queries? If so, I need you!

Subjects are needed to take part in a study concerning the visualization of database queries. Participants will be asked to draw diagrams that represent the execution of database queries or to determine what queries are represented by a set of diagrams. This study will help design a tool intended to help expert and novice programmers to design and debug such queries. The time needed for the study will range from 30 minutes to an hour, and can take place in the Bahen Centre at the University of Toronto or elsewhere in the Greater Toronto Area.

A basic understanding of relational databases and database queries is required. The examples will contain queries in SQL and other programming languages like Ruby or Python. The participants should be fluent/conversant in English.

Participants who complete the study will be entered into a random draw for a $100 Best Buy gift card. The odds of winning this prize are 1 in 30.

If you think you can help Zuzel with her project, drop her a line!

This article also appears in Canadian Developer Connection.

{ 1 comment }

So You Need a Typeface…

by Joey deVilla on April 26, 2010

so you need a typeface flowchart

Got a project and can’t decide on a typeface? This chart is by no means complete, but it might help steer you in the right direction. Click it to see it at full size.

This article also appears in Canadian Developer Connection.

{ 2 comments }

Replica Spanish galleon on fire

Sometimes, you have to do more than just start from scratch. Sometimes, you have to burn the boats.

“Burning the boats” is an expression that comes from a story – some say legend — about Cortes, the Spanish Conquistador (and yes, the subject of Neil Young’s Cortez the Killer). Wishing to guarantee that his men would stay in Veracruz (which he’d just taken over from the Governor of Cuba) and only move forward into terra incognita without retreat, he ordered them to burn the ships that brought them to the New World. It was an extreme measure, but without the distraction of a way home, they committed themselves completely to business of exploring and conquering.

The Original Mac: No Arrow Keys

Bruce “Tog” Tognazzini, former user interface guy at Apple and the company formerly known as Sun, and now member of the Nielsen/Norman Group, wrote about how Apple burned the boats back when they released the original Macintosh in his 1992 book Tog on Interface and more recently in an article on his blog, AskTog.

Original IBM PC and Apple // computers

In 1984, the Macintosh represented a break from the dominant paradigm at the time: the command-line interface. Back then, you’d issue commands to a program these ways:

  • Typing them in
  • Using control-key combinations
  • Using function keys
  • Using the arrow keys to navigate

Software developers at the time had little experience developing for GUIs, which meant that there would be great temptation for them to simply develop apps for the Mac the way they did for other platforms. The software they’d end up writing would be a command-line app that just happened to run on the Mac.

Steve Jobs and Apple’s Macintosh team, an unconventional bunch who were said to have nary a classical computer science degree among them, thought that existing software sucked. I was 16 at the time, and I’d have to agree. In order to prevent straight ports of existing software to the Mac, they decided to “burn the boats” and make it difficult for developers to “go home” and simply rely on the UI techniques from the Old World. The first Mac keyboards didn’t just omit the function keys, they also left out the arrow keys:

Original 128K Macintosh. "See? No arrow, function or control keys."

Tog writes:

That was a big deal. Almost every application then in existence depended on the arrow keys (then called cursor keys) for navigation. With that one stroke, Steve reduced the number of apps that could be easily ported to the Mac from tens of thousands to zero, ensuring that this new computer would have a long and painful childhood.

It’s counterintuitive to want to have your creation go through a long and painful childhood, but there was a method to their madness. In “burning the boats” by getting rid of the function and arrow keys on which developers relied and taking away their “way home”, they forced developers to redesign and rewrite their applications to fit a mouse-driven graphical interface rather than a keyboard-driven command-line interface.

They eventually brought back the arrow keys about a year and a half later. By that point, developers had grown used to developing GUI apps that took advantage of the UI controls and mouse that we’ve come to know and love. The return of the arrow keys at that point would now be a welcome addition and convenience, rather than a dangerous temptation to return to “the old ways”.

It was a bold move, but when you’re making radical changes to the way things are done, bold moves are often required.

Windows Phone 7: No Copy and Paste

Copy and Paste icons

There’s been some talk about Windows Phone 7’s lack of copy and paste. It’s similar to the hue and cry about the original iPhone’s lack of copy and paste, and having been reminded by Tog’s article about the design decisions made for the original Mac, I can see the method to Microsoft’s madness.

“Copy and paste already exists in Windows,” people have said, “why not Windows Phone 7?”

The answer is simple: because Windows Phone 7 apps aren’t supposed to be like Windows apps. For non-enterprise, non-industrial use, the “Windows, but scaled down” approach of previous versions of Windows for phones, which goes under the name Windows Mobile, didn’t catch on (Windows Mobile still rules the roost for compact devices used in enterprises and industries, and will be supported for years to come). Hence Albert Shum’s completely different-from-the-desktop, and even different-from-other-phones Windows Phone 7 interface, which went by the codename “Metro”.

Windows Phone 7 hubs: music+video, people, pictures, office, games

The use of copy and paste implies a keyboard-centric user interface, which isn’t what Windows Phone 7 is about. People often use their smartphones one-handed, with only their thumb to access the touchscreen. Windows Phone 7’s interface takes this usage into account, which is why it’s sensor-centric, and applications, should get their information from touch, gestures, accelerometers, location and other sensors where possible. By not including copy and paste in the first release, the Windows Phone team is “burning the boats” and asking developers “How do you write apps so that they don’t need intricate more-suited-to-the-desktop operations like copy and paste?”

(And yes, copy and paste will eventually find its way into Windows Phone 7, just as the arrow keys, function keys and even right-clicking found their way into the Mac.)

The same could be said for many other things that were purposely excluded from Windows Phone 7, such as the compact edition of SQL Server that was part of Windows Mobile. If you think about it, this design decision forces you to build apps so they store and retrieve data from the network, which makes sense, since phones are devices that network with both cellular and wifi.

Windows Phone 7 represents a radical shift in the way Microsoft stuff works, from a very minimalistic look to its task-centric organization. In order to make sure that people built apps that fit it, the Windows Phone 7 team had to burn the boats. It’s a bold move, but it’s the right one.

{ 19 comments }

MIX10 Thoughts: Design, Windows Phone and Bill Buxton

by Joey deVilla on March 23, 2010

Joey deVilla and Bill Buxton posing on the dance floor at LAX nightclub in Las VegasMe and Bill Buxton at the MIX10 Attendee Party last Tuesday night.

There’s one reason I’m particularly excited about Windows Phone 7 Series. The radically reworked look and feel is the surest sign that the company is really beginning to understand design and is willing to start from scratch (a risky and pricey proposition) to get it right. It would appear from Microsoft’s Principal Researcher Bill Buxton’s interview in The Register that I’m not the only one who thinks this:

"We kinda changed the water that we drink, in the sense that all through the design community within the company we talk, and we have a common goal in terms of trying to bring a certain change of sensibility," he said.

"For me it’s not even about the phone, but what’s interesting is that it’s the first product in the company with critical mass that’s embraced this … it will have an impact on other parts of the company."

If (or better still, when) you start building Windows Phone applications – or hey, any kind of application — I hope that you’ll follow the spirit of “Metro” (the codename for the design philosophy behind Windows Phone 7) and keep it in mind. Yes, it’s absolutely important to know the Silverlight and XNA APIs as well as how to read the touch sensors, GPS, accelerometers and so on, but it’s just as important to design your applications around the people who’ll use them. That means understanding your users, how they’ll use what you’re making, knowing how to give them what they need as quickly and unobtrusively as possible and delighting them. Yes, “a pretty interface” is included in all that – and there’s research to suggest that beautiful interfaces work better –but looks are merely part of the design equation.

Some Design Sessions from MIX10

Want some interesting lunchtime viewing on design? Look no farther than these two videos.

The first is Bill Buxton’s MIX10 session, simply titled An Hour with Bill Buxton, a conversation about design:

Get Microsoft Silverlight
Don’t have Silverlight? Get it here or download the video in
WMV, WMV (High) or MP4 format.

If you haven’t the time to watch the video of Buxton’s presentation, check out this hit list of metaphors complied by Sharon Chan at Microsoft Pri0.

If you’re planning on getting into Windows Phone 7 design, you’re going to want to learn the “design language” – not a programming language, but the guiding principles and philosophies behind the new user experience – behind it. Here’s the MIX10 presentation on that topic, Designing Windows Phone 7 Series with Albert Shum, Michael Smuga and Chad Roberts:

Get Microsoft Silverlight
Don’t have Silverlight? Get it here or download the video in
WMV, WMV (High) or MP4 format.

A World Without Design

Here’s a little something extra for those of you who like to think about design and user experience:

"A World Without Design": The same rock, described as a hammer, doorstop, paperweight and so on.

I whipped up this graphic, modelling it after a poster I remember seeing many years ago. I can’t remember what it was for – a museum, art gallery or exhibit, perhaps? – but I remember thinking that it was right on the money. If you remember the original poster and what it was for, please let me know, either via email or the comments!

This article also appears in Canadian Developer Connection.

{ 0 comments }

Windows Phone 7 Series: Now That’s More Like It!

by Joey deVilla on February 15, 2010

Windows Phone 7 Series generic phone

A New Windows for the Phone

Ever since joining The Empire, I’ve been saying that Windows Mobile needs to go back to the drawing board. While there was good technology lying in its innards – mobile versions of the .NET framework, SQL Server and Office – treating the mobile form factor as “the desktop, but much, much smaller”, was the wrong approach. In the meantime, the Esteemed Competition were doing the right thing: designing their phones’ OS features and interface from the ground up rather than attempting to force-fit the desktop UI into a pocket UI.

Today at Mobile World Congress in Barcelona, Microsoft previewed the latest in a series of steps forward – consider Xbox to Xbox 360, Windows Vista to Windows 7, Live Search to Bing – there’s now Windows Phone 7 Series.

(The name’s a bit long. Whoever does the naming at Microsoft corporate HQ must get paid by the syllable.)

A Quick Look at Windows Phone’s Experience

A good starting point is this video, which covers Windows Phone’s features in three minutes, thirty seconds:

You can take an interactive tour of the UI at the Windows Phone 7 Series site:

Screenshot of the Windows Phone 7 Series site's home page

A Closer Look at the Windows Phone Experience

Over at Channel 9, Laura Foy has posted her interview with Joe Belfiore, VP Windows Phone 7 Program Management, who gave her a walkthrough of the goodies in Windows Phone (the video is 22 minutes, 18 seconds):

Get Microsoft Silverlight

Some quick notes from the video:

  • There are three mandatory hardware buttons, which are context-sensitive:
    • Back
    • Windows (the “Start” button)
    • Search
  • The screen is a capacitive touch-screen, capable of supporting multi-touch
  • The Start menu is built up of tiles: little block representing the information and features that you care most about
    • You can add your own custom tiles; Joe shows a “me” tile linked to his Facebook profile
  • A browser with:
    • Snappy performance
    • Support for multitouch actions such as pinch zoom, double-tap to zoom and finger drag
    • Very readable text, that to sub-pixel positioning in HTML
    • Phone number recognition in HTML documents; touch them to dial them
    • Street address recognition in HTML documents; touch them to get a map
    • Multiple tabs
  • The “People Hub”
    • Aggregates Exchange, Hotmail, Gmail, Yahoo! Mail and other mail contacts
    • Provides a live feed of your contacts
  • Context-sensitive search:
    • Press the “Search” button while in the People Hub, and you search your people list
    • Press the “Search” button while in the Start menu, and it runs a web search
      • Based on your query, it knows whether to give you a web search result or a local search result
      • In the demo, Joe does a search for pizza and gets a map and results for pizzerias near him, and a quick pan over to adjacent pages yield directions and reviews
      • A tap on “nearby” yield the locations of useful things like parking, ATMs and so on near the selected pizzeria
      • In another demo search, Joe does a search for “Avatar” and it returns a list of nearby theatres and times for the movie Avatar; a quick pan to an adjacent page yields the results for local business and places with “Avatar” in the name
  • Email:
    • Easy pivoting between unread, flagged and urgent emails
    • A caching system prevents you from seeing the dreaded “loading” screen
    • Press “Search” within email and you perform a search of your email messages, by subject, text and so on
  • Rotation: you can operate the phone in “portrait” or “landscape” mode
  • Calendar:
    • Support for both work and personal calendars
  • ActiveSync works in the background and keeps the phone synced with email, contacts and calendar
  • User-customizable UI colour schemes
  • The “Pictures Hub”
    • Gallery: Lets you browse all the pictures on your phone
    • Mosaic: Recent and favourite pictures
    • What’s New: New photos from your social networks
    • Camera roll: A folder for photos taken with your phone
    • Support for photo albums from Facebook and Windows Live, which you browse as if they lived right on your phone
  • Music and Video
    • History: Most recently played music and videos
    • New: New music and videos added since the last sync
    • Zune HD-style marketplace searching and support for Zune subscriptions with unlimited music plays
  • The “Me” tile
    • Lets you update your status on places like Facebook
    • Nice little typing features like auto-spelling-correction and a special soft keyboard for emoticons
  • The UI concept: Windows Phone is task-centric, not app-centric, with a hub associated with each: people, photos, media
  • There’s also a games hub, which ties into Xbox Live
  • Third-party applications and games? Wait…

Wait a Minute…What About Third-Party Apps and Games?

"MIX10: The Next Web Now" logo buttonCan you wait a month?

Here’s the deal: the announcement at Mobile World Congress was about showing what Windows Phone can do. As for what’s possible on the developer front, it’ll all be announced at the MIX10 Conference, which takes place from March 15th through 17th in Las Vegas.

There will be a dozen sessions at MIX10 for Windows Phone, and they promise to be quite interesting. I’ll be at MIX10, and will blog what I learn from these sessions when they take place.

You can save $200 off the price of MIX10 registration if you register before February 21st, so if you want to get in on the ground floor with Windows Phone and save some money, register now!

What the Tech Press is Saying

Pretty good stuff, actually. Rather than bury you with links to a zillion blog entries filed from Mobile World Congress, I thought I’d pick two of the big tech blogs, Gizmodo and Engadget:

Here’s what Gizmodo has to say about the new Windows Phone:

It’s different. The face of Windows Phone 7 is not a rectangular grid of thumbnail-sized glossy-looking icons, arranged in a pattern of 4×4 or so, like basically every other phone. No, instead, an oversized set of bright, superflat squares fill the screen. The pop of the primary colors and exaggerated flatness produces a kind of cutting-edge crispness that feels both incredibly modern and playful. Text is big, and beautiful. The result is a feat no phone has performed before: Making the iPhone’s interface feel staid.

If you want to know what it feels like, the Zune HD provides a taste: Interface elements that run off the screen; beautiful, oversized text and graphics; flipping, panning, scrolling, zooming from screen to screen; broken hearts. Some people might think it’s gratuitous, but I think it feels natural and just…fun. There’s an incredible sense of joie de vivre that’s just not in any other phone. It makes you wish that this was aesthetic direction all of Microsoft was going in.

Here are Engadget’s impressions, after having some hands-on time with Windows Phone:

The design and layout of 7 Series’ UI (internally called Metro) is really quite original, utilizing what one of the designers (Albert Shum, formerly of Nike) calls an "authentically digital" and "chromeless" experience. What does that mean? Well we can tell you what it doesn’t mean — no shaded icons, no faux 3D or drop shadows, no busy backgrounds (no backgrounds at all), and very little visual flair besides clean typography and transition animations. The whole look is strangely reminiscent of a terminal display (maybe Microsoft is recalling its DOS roots here) — almost Tron-like in its primary color simplicity. To us, it’s rather exciting. This OS looks nothing like anything else on the market, and we think that’s to its advantage. Admittedly, we could stand for a little more information available within single views, and we have yet to see how the phone will handle things like notifications, but the design of the interface is definitely in a class of its own.

(In another article, Engadget simply summed it up with “Microsoft is playing to win”.)

Watch this Space!

"Counting Down to Seven" badgeWe’ll have more announcements about Windows Phone over the next few weeks, so keep an eye on this blog!

This article also appears in Canadian Developer Connection.

{ 4 comments }