Accessing embedded program in untestable mobile environment : experience of a trustworthiness approach

Comparing actual output with the expected output of some controlled input is a fundamental principle of program correctness testing. However, in some situations, the input is uncontrollable or even undetectable during testing, it is impossible to decide the expected output or the test oracle. Or in some situations, the output may be indeterministic or unpredictable, hence it is impossible to compare the output with the expected one. We encountered these problems and could not conduct normal program testing when we developed programs to extract network data from various mobile stations in the Mobile Location Estimation System project. Instead we analyzed the program output and challenge it against the intrinsic properties, the environment, the program itself, and the application results to find evidence that the output is suitable to be used for the planned purposes. In short, we accessed the trustworthiness of the programs. In the development of our mobile software, it was common that different programs of the same specification had to be developed for mobile stations of different models. These different implementations provided another source of reference for trustworthiness assessment. Our experience of applying the trustworthiness approach to developing software for extracting network data from mobile stations is reported in this paper.