Workshop  | Bret Pettichord

Fourth Annual Austin Workshop on Test Automation (AWTA)

January 25-26, 2003 -- Austin, Texas

Building and Using Test Interfaces

Workshop participants were: (standing) Chris Sepulveda, Michael Silverstein, Brian Marick, Andy Tinkham, Elisabeth Hendrickson, Cem Kaner, (seated) Rob Mee, Bret Pettichord, and Ward Cunningham.

Call for Participation

Based on early discussions, we have decided to broaden the scope of the workshop. We are interested in discussing various approaches for creating and utilizing programming interfaces to product software to facilitate testing. We'll discuss IPC-based interfaces as well as other kinds of API's, language-based interfaces, and built-in test fixtures. If you were going to create an interface to directly support product-level testing, what would it look like?

Brian Marick's position paper. "It touches on architectures to support product-level tests, the way I use test automation to explore design, and my increasing inability to tell the difference between acceptance and unit tests."

What interfaces should we use to test software? One tradition uses Graphical User Interfaces (GUIs), typically with GUI testing tools. Another tradition uses the programming interfaces provided by the methods and functions of the software under test -- often used for unit testing. This workshop will explore other methods for providing interfaces to software to support testing. One alternative we'll explore is creating and utilizing interfaces for inter-process communication (IPC) using the many interface technologies that are maturing, such as XML-RPC, XMI, SOAP, COM, or Java RMI. A related area of interest is the the practice of embedding interpreters into products to support testing scripts. We are also interested in the use
of API's, test parsers, and embedded test fixtures.

Possible topics include:

  • How have you created and used such interfaces?
  • What are technical challenges to creating such interfaces?
  • What are social or organizational challenges to utilizing such interfaces?
  • What are the benefits and drawbacks to these interfaces, as compared to the alternatives?

Workshop Goals

  1. Exchange concrete techniques and approaches regarding creating and utilizing such interfaces.
  2. Encourage the development of published reference implementations.
  3. Understand how using such interfaces for testing affects the roles of developers, testers and other members of a development team.

Attending the Workshop

Participation in the workshop is by invitation based on a position paper. The workshop is limited to 15 participants. Your position paper should have two parts.

  1. Experience Report. Describe your experience using IPC or embedded-language interfaces for testing. Relevent experience may include other kinds of interface technologies than those listed in this call.
  2. Position Statement. State something that you think people should know. This may be a technique for providing test interfaces, a reference implementation you would like to demonstrate, a challenge that such interfaces may run into, or a reason for why further investigation is warranted.

So far we have received inquiries both from talented testers who are concerned that they may not have sufficient technical background and from perceptive developers who are concerned whether they will fit in. The topic of this workshop is challenging because it deliberately cuts across traditional boundaries. We encourage submissions from interested testers and developers who are open-minded and interested in learning from others. We expect that some participants will only have tangential experience with the topic under discussion.

Position papers should be between one and three pages long. After acceptance, they will be posted to a private web site that will be shared by other participants. Thus, a web-based format is preferred. We can publish text, HTML and PDF. We can also generate PDF from MS Word files.

Position papers should be submitted to Bret Pettichord ( ). Papers will be reviewed on a rolling basis, with replies in three days or less. The workshop website will indicate when the workshop has been filled.

What Will Happen at the Workshop

The workshop will be organized as a moderated discussion, following the format of the Los Altos Workshop on Software Testing .

Participants will be selected to present particular techniques and experiences of using test interfaces. The workshop will explore the techniques and the conditions that favor or disfavor the technique.

We expect that subthemes will emerge from this discussion. If suitable, we hope to for subgroups to explore these themes and report back to the larger group.


Participants are responsible for own travel and lodging. The workshop expenses, including some meals, will be covered by the workshop sponsors.

The Organizers

Bret Pettichord is the founder of the Austin Workshops on Test Automation. He is an independent consultant and trainer specializing in testing and test automation. He has had several opportunities to test software using test interfaces and would like to see more developers provide them and more testers use them. He is co-author of Lessons Learned in Software Testing and editor of the

Brian Marick specializes in code-based software testing. He is currently developing a reference application that has an IPC interface for testing. He is the author of The Craft of Software Testing and technical editor for STQE magazine.

Cem Kaner is professor of Computer Sciences at Florida Tech, where he's developing a curriculum for training test architects. He is the founder of the Los Altos Workshop for Software Testing (LAWST), and lead author of Testing Computer Software and Lessons Learned in Software Testing .

Time Frame

Sat Jan 25, 9 am to 5 pm
Sun Jan 26, 9 am to 3 pm

The workshop will start promptly at 9 am on Sat Jan 25. Participants from out of town should plan to arrive the night before. There will be a welcoming dinner on Friday Jan 24; participants are welcome to invite family, friends and colleagues to dinner.

We expect all participants to attend for Saturday from 9 to 5 and Sunday from 9 till noon. Prospective participants who won't be able to attend for this time should so indicate when they submit their position papers. Usually most participants plan to attend until 3 pm on Sunday. We'll have the space until 5 pm; often a subgroup will be active until then.


The workshop will be located at the Homewood Suites in Austin, Texas. We used the same location for AWTA3 and the participants enjoyed the cozy setting.

Homewood Suites
10925 Stonelake Blvd
Austin, TX 78759

Upon invitation, mention Pettichord Consulting to get a group rate of $79 a night, available through Jan 5.

Why Position Papers?

Previous AWTA workshops have not used position papers. Instead, invitation was based on acquaintance, referrals, interviews and occasionally on email exchanges. The use of position papers formalizes this process. We have used them successfully in other workshops and are introducing them to AWTA for the following reasons:

  1. Increase the pool of potential applicants. The previous method made it difficult to invite people whom the organizers were not already acquainted with.
  2. Assure understanding of workshop goals. Confusion regarding the goal of the workshop can be identified and corrected early.
  3. Jump start the workshop. Traditionally, the workshop has opened with each participant introducing him- or herself and the issues they bring to the table. This easily takes an hour that the position papers save. All participants are expected to have read the position papers before the start of the workshop.
  4. Help plan the agenda. The workshop agenda is based on the issues and experiences of all the participants. Getting this information in advance -- rather than during the opening introductions -- makes it easier for the organizers to plan an agenda that makes the most of what the participants bring to the table.
  5. Set expectations regarding participant contributions. These workshops expect some kind of contributions from all participants. The position papers provide an example of this and allow prospective participants to demonstrate their ability to describe a position and share it with others. The goal of these workshops is continued public engagement and presentation.

In short, position papers help assure that we get qualified participation and streamline the workshop so that we can get the most accomplished during a limited time. This policy supercedes the previously policy whereby previous AWTA participants were automatically invited. However, they will continue to get early notification of the call for participation.   |   Copyright 2002, Bret Pettichord