Windows 8 app development and testing

As they continue to innovate on the operating system platform with Windows 8, they also enhanced the toolset available in Visual Studio to increase your productivity in testing your Windows 8 apps. Ensuring a top notch quality of your app in this new modern OS environment through thoughtful design and testing will help increase the success of your app. In a previous blog post on Testing Metro style apps in Windows 8, we highlighted some of the key areas you need to take into account for building a high quality Windows 8 app. In this post, we explore a few capabilities in Visual Studio 2012 that will make the testing and verification of your app easier.

How does my app look on Windows 8 devices?

While the success of any app hinges on the value it offers to its customers, an app that follows Windows 8 design guideline to provide a polished look and feel will delight the end users. To reduce your reliance on physical devices to verify the app’s user experience, Visual Studio 2012 provides a simulator so you don’t have to connect to a tablet device all the time during development. The simulator also reduces your need to have physical devices of every form factor at your disposal for testing. Your testing benefits in these areas where the simulator enables quick and convenient verification of your apps’ look and feel with just a few mouse clicks:

• Screen size, resolution and aspect ratio
• App layout
• Screen orientation

Figure 1. Visual Studio simulator displaying the Windows 8 start screen.

Windows 8 apps run on devices of multiple form factors. To help you test your app’s behavior in different screen sizes and resolutions, Visual Studio simulator provides 7 most common combinations of those two factors that cover from the minimum supported resolution to a 27” desktop monitor. Figure 1 illustrates an example of Windows 8 start screen shown in the simulator at the default resolution of 1366x768 and screen size of 10.6”. One of the first things to verify is that at all times the layout of your app is not clipped or misaligned vertically or horizontally at lower resolutions. On the other hand, you don’t want the app to leave a large amount of blank space at higher resolutions either. If your app’s layout extends beyond the screen, the scroll bar should show up appropriately if a user is interacting with the app using a mouse. These are examples of the type of verification you can easily accomplish by changing the simulated resolutions on the fly in the simulator and visually examining the behavior of the app.

If your app uses a fixed layout, you need to test whether it scales correctly to fill the screens of large sizes and make sure all the visual elements also scale proportionally. If your app has an adaptive layout, the simulator allows you to quickly verify that the app’s visual elements are dynamically rearranged as you intended when screen size or resolution changes.

The simulator can be rotated or flipped to see how your app behaves when a user changes the orientation of the screen. This comes in handy if you want to test such an app to see if it is correctly optimized for the portrait mode and the landscape mode. The simulator doesn’t eliminate the need to fully verify the app on physical devices, because it doesn’t simulate all the accelerometer sensitive capabilities such as tilt, but it will save you a lot of time in exposing potential problems in your app’s response to the changing of the orientation of the displaying device.

How does my app respond to touch on Windows 8 devices?

The simulator allows you to interact with your app using mouse or using simulated touch on your development computer. The latter is particularly handy if you don’t have a touch-capable device available for testing. Swipe, pinch to zoom and rotation gestures are all supported.

You typically launch your app in the simulator directly from within the Visual Studio through the debugging action. After the app is launched, the simulator stays running, which allows you to also exercise the app outside of the debugging context. For example, you can watch the activation behavior when a user taps your app’s tile or the termination behavior when a user swipes from the top to the bottom of the screen. You can take advantage of the touch gestures that the simulator provides to perform these actions and all the touch related testing and verification. Testing Metro style apps in Windows 8 has a detailed list of testing considerations related to touch. Refer to the Touch section in that blog post. If you want to learn more about the Visual Studio Simulator such as how to use the touch emulation, take a look at Running Windows Metro style apps in the simulator.

Unit testing your Windows 8 apps

The ability to unit test your code is an important aspect of any development cycle. Unit test support has been shipping in Visual Studio for several releases. In Visual Studio 2012, we have streamlined the user experience with a new unit test explorer and created an extensible test runner that is independent of the underlying unit test framework. Now you can plug in your favorite unit test framework such as NUnit, XUnit or MbUnit, or you can continue to use MSTest.

  Figure 2. New Project dialog in Visual Studio 2012 shows the template for the
Unit Test Library project for the apps written in C#.

For the Windows 8 apps written in C#, VB.net or C++, Visual Studio now provides a unit test library project that follows the same workflow you use for developing a desktop app (see Figure 2). You will enjoy the improvements and many of the same features you are familiar with. Under the hood, the unit test support takes care of running your tests in the app container model and abstracts all the complexity away. The unit test project has its own appxmanifest file. This is helpful because in many cases such as when you are testing the code that handles location service or internet access, for the tests to run, you need to edit the manifest file to specify the capabilities that match your app’s requirement, as illustrated in Figure 3. Alternatively, if the part of the code path under testing doesn’t need a capability, you can group such test cases in a separate unit test project and deliberately exclude a capability from it to improve the modularization of your code.

Figure 3. Package.appxmanifest file of a unit test project opened in the Manifest Designer. 

In keeping with our commitment to quality, we made the improvements and new functionalities in unit testing available in all the SKUs of Visual Studio 2012, including Visual Studio 2012 Express for Windows 8. For more details or to learn how to effectively unit test your Windows 8 app, see Creating and running unit tests on a Windows Metro style app and this unit testing talk from the //BUILD conference.

Exploratory testing your Windows 8 apps

Windows 8 apps are designed to excite consumers and businesses alike. If you work on an app in an enterprise environment, and are used to a more structured development and testing process, Visual Studio 2012 provides a manual testing tool, Microsoft Test Manager, which in combination with Remote Tools for Visual Studio, allows you to conduct exploratory testing and structured manual testing of your app’s functionality on all types of Windows 8 devices. Microsoft Test Manager is part of the Visual Studio Premium and Ultimate SKU and the Test Professional SKU. In addition to the seamless integration with TFS for easy bug filing and test case management, the manual testing tool in Visual Studio 2012 simplifies testing of your app on a remote device. During the testing process, the tool can capture screen shots, action logs and event logs from the remote device when you discover a bug in your app. It provides a two-way communication between your main workstation and the connected device that guides you through the steps of deployment of your app packages to the targeted testing device, acquisition of developer license on the device, and the execution of test steps.

With TFS integration, the manual testing tool enables a continuous workflow in a team environment between testing and development of your app. For example, the info captured during exploration helps you reproduce the bug and improves the understanding of its nature. When a new app package becomes available with bug fixes or feature enhancement, it can be easily picked up and deployed to the same Windows 8 testing device for further verification from within the manual testing tool.

For more info, see Visual Studio 11 Beta: Manual Testing of Windows Metro Style Apps. Manual Testing Of Windows 8 Metro Style Applications provides a detailed walkthrough.

Conclusion

Windows 8 is an exciting platform for developers and users. We hope that this blog is a useful introduction to the tools shipped in Visual Studio 2012 that you can start using right away to test the Windows 8 app you are working on. We look forward to hearing your feedback to see how we can continue to make that part of your life easier. Don’t forget to leave comments on this post to share with us and the community tips, ideas, and personal experience on how you have been testing your apps.