How perfect my implementation should be?

In the program analysis field, we built a lot of software tools. Building a mature program analysis tool can take significant engineering efforts. In your limited time graduate studies, you do not want to swamp yourself into massive tool building.  For writing a research paper, the implementation should be driven by experiments. Remember that your primarily goal here is to demonstrate your new algorithms and methodologies work. During or even before the implementation, you need to think about the following questions:

What are the goals of your experiments? For each goal, what are the research questions you aim to answer? For these goals and the research questions, what types of data you want to collect? what are the metrics? For each type of the data, what types of experiments you should design? Finally, what implementation needs to be done to support these experiments? Then how the data can be presented to the readers? tables? figures? or processed via statistical methods?

Note that some of the tools we developed can potentially be reused by other researchers in the community. For release purposes, we then need to do more implementation to better support readability and usability of the tools. Once the tools are released, you can choose maintain the tools and respond to the users’ questions, like Diakon; or you can just release a tool and put a note “the users are responsible for bug fixes”. Of course, the first approach will attract more users but require higher maintenance effort.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s