Testing Summary

From Forum Nokia Wiki

This is a place holder for all testing, signing, capability request, developer certificate, etc related material.

The purpose of this page is to have a well organized table of contents to guide into reading relevant material within this Wiki.

Introduction

The Symbian Signed or Symbian Signed for Nokia testing may usually be considered (from “normal” development point of view) as external or additional tasks. This should actually not be so as if thinking them as such they indeed will be that: external, additional, costly burdens. For certain applications Symbian Signed for Nokia testing is an integral part of the whole application development and deployment – only the developer (you) does not know that yet.

Testing and signing issues should be among the first to consider in a new Symbian project, as they will be major obstacles down the path after you have solved or fixed other critical issues. If testing and signing are confronted as late as when they become the necessary next steps i.e. after writing the code and after internal testing, then they may pose unplanned changes to your code, additional cost in multiple testing rounds by an external test house, and, likely a delay to your publishing date. Early on, get aquainted with both the Symbian Signed test criteria and the Symbian Signed for Nokia test criteria. Incorporate the methodology of these criteria into your design from the start.

Symbian Signed test criteria: https://www.symbiansigned.com/app/page/overview/testcriteria

Symbian Signed for Nokia test criteria: http://www.forum.nokia.com/document/Nokia_Test_Criteria_for_Symbian_Cpp_Applications/

Forum Nokia is a wealth of information on the actual signing processes, you just need to know where to find this information. Start with http://www.forum.nokia.com/testing and http://www.forum.nokia.com/platformsecurity.

Platform Security

Platform security and signing go hand in hand. Here are the basics of Platform Security and the capabilities paradigm defined by Platform Security (introduced in Symbian OS v9; S60 3rd Edition).

Platform Security introduces obligatory application signing, and voluntary application certification. Read more about different certificates that are used to sign applications.

Design (for testing)

Testing should be addressed already in the design phase, as the Symbian Signed for Nokia test cases take into consideration many UI Style Guide considerations, as well as some usability issues. Any pointers to help with usability and UI related testing, can be gathered under UI test criteria.

Tools to help with testing

Remote Device Access (RDA) might be helpful when you do not have a specific target device at hand at your location.

Bear in mind that the software that is used with the RDA devices must also be signed. During the development phase you will need to sign your installation package with a DevCert and for that you will need to know the IMEI of the RDA device. For further instructions on how to proceed, please refer to this info page: http://www.forum.nokia.com/main/technical_services/testing/rda_instructions.html

To access RDA, browse to this location: http://www.forum.nokia.com/main/technical_services/testing/rda_introduction.html

Carbide.C++ tools & plug-ins

Carbide product family has some useful tools for application testing. These have quite good documentation and user guides so those won't be replicated here, but links to some further information.

Carbide.c++: Performance Investigator

Other tools

DumpSIS - for purging the contents of a signed/unsigned SIS file. Ships with the S60 3rd ed, FP1.

Hands-on help: consultancy

How to apply for external testing consultancy?

Testing Phase

Now I'm done testing with the emulator. How to get a DevCert to test with sensitive capabilities on a real device?

When applying for a DevCert, an important thing to consider is "does my application need manufacturer or platform capabilities?" if it does, you need to submit your application via a different link within the Symbian Signed portal (www.symbiansigned.com). This portal combines the previously separated "capability request" and "request form", that were being submited to different addresses. Now one address, one portal is all that is needed to submit the same information content.

When submitting, one has to upload a csr file (this will dictate which capabilities you request, and what IMEIs i.e. devices you will have this certificate to work on). Then, according to this csr file you must give a technical reason to each sensitive capability. This information was previously submited on the separate "request form". Try to be as detailed with the technical need as possible, otherwise the capability board will have to contact you to query this information.

The sensitive capabilities will not get granted easily i.e. without a solid technical need, so it may be worth while to check out these pages on both "requests that will not get approved", and for a categorization of applicaiton types that most likely will need some of the sensitive capabilities:

Capability denial criteria

Sensitive Applications

Signing Phase

Now, submitting an application to a test house for final testing and certification. In your own tests you might have had "collisions" with the Symbian Signed for Nokia test criteria that, by your application intended functionality, should be ignored when running tests on you application. Nokia and the test houses call these waiwers.

Read more on requesting waivers.
Read more on satisfying Symbian Signed and Nokia testing criteria.