Git and Gerrit workflows for long-running topic branches

At Optemo, we frequently have a topic branch that runs for days or weeks before we are ready to merge it in to master and deploy it to the production site. When we are finally ready to merge such a branch, we would like to have a code review in Gerrit which captures all of the changes that will be added to master.  This page describes two possible approaches we have found for achieving this.  The approach we prefer, especially when multiple programmers are working on the same topic branch, is Workflow 1. However, if only a single programmer is working on the topic branch and the branch is not expected to be very long-lived, Workflow 2 is also an option.

We are by no means Git and Gerrit experts yet, so let me know if you have any comments or alternative approaches to suggest.

Workflow 1: Using merge

In this workflow, you use merge when you need to update the topic branch with the latest changes in master:

1. git checkout <topic branch>
2. git fetch origin
3. git merge origin/master

You follow the same process when you need to update your topic branch with changes made to this branch by another programmer:

1. git checkout <topic branch>
2. git fetch origin
3. git merge origin/<topic branch>'''

When you are ready to push the changes for code review, you merge your topic branch in to master and use the --squash option to create one big commit.

1. git checkout master
2. git pull origin
3. git checkout -b <temp branch>
4. git merge --squash <topic branch>
5. git push origin HEAD:refs/for/master

Note that once the change has been submitted by the reviewer and merged in to master, you should stop using the old topic branch. New work should happen in a new topic branch created off the latest version of master.

Workflow 2: Always rebase on top of master

In this approach, whenever you need to update the topic branch with the latest changes in master, you follow the following steps:

1. git checkout <topic branch>
2. git fetch origin
3. git rebase origin/master

This takes the changes in the topic branch and applies them on top of master's HEAD.

When you are ready to submit the topic branch for code review, you follow a similar process. You rebase the topic branch on master, then push the changes in the topic branch for code review. You can do an interactive rebase using the -i option, which will allow you to squash all the commits on the topic branch together in to one big commit:

1. git checkout <topic branch>
2. git fetch origin
3. git rebase -i origin/master
4. git push origin HEAD:refs/for/master

Note: If you are following this workflow, it is important that you do not use merge to update the topic branch with the contents of master. If you do, and you later perform a rebase prior to submitting your changes for code review, you may find that you have to redo work you did in earlier merges. Specifically, you may have to manually resolve conflicts that you previously resolved.

Eclipse Plugins that need early startup

A pattern for Eclipse plugins that need early startup, and also have views (which may or may not be part of the active perspective on startup):

1. Implement the org.eclipse.ui.startup extension point in a separate class, as the docs recommend, but don't actually do any work in the earlyStartup() method.
2. Define a finishInit() method in your AbstractUIPlugin which is going to do the actual work of initializing your plugin. Also define an initComplete boolean field. Check this field at the start of finishInit(), and set it at the end, to ensure that the code inside finishInit() is only executed once.
3. In the start() method of your AbstractUIPlugin, first initialize your static singleton instance field.  Then invoke finishInit() on the UI thread with PlatformUI.getWorkbench().getDisplay().asyncExec(). (Why? If your view is not in the active perspective, then start() will be called by a non-UI thread. The effect of this is that the workbench is not yet fully initialized when your start() method is called.)
4. In the createPartControl() method of your views, first call finishInit() on the singleton AbstractUIPlugin instance. In the case where your view is present in the active perspective, createPartControl() will be called before the async call to finishInit() is processed on the UI thread. So you call it at the start of createPartControl() to make sure plugin initialization completes before you begin creating your views.

Reducing the effort of creating JUnit test oracles

So you've gone to the trouble of defining a set of inputs that exercise a particular path through your code. Now you need to write the oracle: the chunk of code that verifies the result is what you expect. This can often be just as much effort as carefully choosing those test inputs. It can also be quite error-prone -- if you're like me, you find a lot more bugs in your tests than in the code your testing!

Here's an approach that can save some time in certain cases. You serialize the result of the test to a text file, then verify (by inspection) that it matches your expectations. For subsequent runs of the test, your test driver reloads the previous result and compares it to the result obtained on the current run.

There are some issues, of course. The test can be a bit more fragile -- if any field of the test result changes, the test will fail. But then updating the oracle will be quick -- just need to dump the new result to a text file, verify it again, then make it the new expected result.  Diff utilities can speed up the re-verification process.

Serialization of the result can get tricky, especially if your result is a tree of objects. Some objects in the tree may not lend themselves to serialization. For others, it may not be appropriate to consider them part of the test result. In cases like this, you may need to dig in to the documentation of the serialization library you are using. The approach I am proposing here might just not be feasible in some cases.

A third concern is that in inspecting the serialized result you could miss problems that you would catch if you manually wrote the oracle. This is subjective, but after using this technique for a few months, I feel that inspection can catch most problems. And, of course, manually constructed test oracles can have bugs that go undetected too!

Here is a JUnit test driver that implements this approach using the Jackson JSON library for serialization and deserialization. Note that with the setup below, only public fields and getters will be checked (but Jackson can be configured to look at private fields as well).

import java.io.IOException;
import java.io.InputStream;
import java.io.StringWriter;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;

import org.codehaus.jackson.JsonFactory;
import org.codehaus.jackson.JsonGenerator;
import org.codehaus.jackson.JsonNode;
import org.codehaus.jackson.JsonParseException;
import org.codehaus.jackson.JsonParser;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.util.DefaultPrettyPrinter;

import static org.junit.Assert.*;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class AutoGeneratedOracleExampleTest {
    private JsonNode testData;
    private String testName;
   
    @Parameters
    public static Collection<Object[]> getParameters() {
        return Arrays.asList(new Object[][]{
                { "Test1" },
                { "Test2" },
                });
    }
   
    public  AutoGeneratedOracleExampleTest (String testName ) 
            throws JsonParseException, IOException {
        this.testName = testName;
    }
   
    @Before
    public void setup() throws JsonParseException, IOException {
        InputStream testDataStream =
            CodeElementExtractorTest.class.getResourceAsStream(
                "AutoGeneratedOracleExampleTestData.json");
        if (testDataStream == null) {
            throw new IOException("Test data file not found");
        }

        try {             
            JsonFactory factory = new JsonFactory();
            JsonParser parser = factory.createJsonParser(testDataStream);
            parser.setCodec(new ObjectMapper());
            testData = parser.readValueAsTree();
        } finally {
            testDataStream.close();
        }
    }
   
    @Test
    public void runTest() throws IOException {
        Object result = ... 
       
        JsonNode expected = getExpectedResult(testName);
        assertEquals("formatted actual: " + getJsonString(result), 
            expected, getJsonNode(result));
    }
   
    private JsonNode getExpectedResult(String testKey) {
        return testData.get(testKey);
    }
   
    private JsonNode getJsonNode(Object obj) throws JsonParseException, IOException {
        JsonFactory factory = new JsonFactory();
        JsonParser parser = factory.createJsonParser(getJsonString(obj));
        parser.setCodec(new ObjectMapper());
        return parser.readValueAsTree();
    }
   
    private String getJsonString(Object obj) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        StringWriter writer = new StringWriter();
       
        JsonGenerator jsonGenerator =
            mapper.getJsonFactory().createJsonGenerator(writer);
        jsonGenerator.setPrettyPrinter(new DefaultPrettyPrinter());

        mapper.writeValue(jsonGenerator, obj);
        return writer.toString();
    }
}

Browser extensions that need to call native code

I've been working on a browser extension for Firefox and Chrome that needs to invoke some native code.  It turns out that the only way to do this in Chrome is through an NPAPI plugin (NaCl is not an option here because the native code needs to perform some privileged operations).  Luckily there is the Firebreath project which makes it pretty easy to create a cross-platform NPAPI plugin for Mac, Linux, and Windows.  You can load the plugin once in the background page of your Chrome plugin and then your content scripts can communicate with it through message passing.

In Firefox, however, there are challenges to calling an NPAPI plugin from an extension.  The two options are (a) to load the plugin in to the DOM of the browser itself; or (b) to inject the plugin in to the DOM of each page as it is loaded.  Option (a) seems like a bit of a hack, and (b) has security issues (since scripts on the page could then invoke the plugin).

A better option in Firefox is to use the js-ctypes API to call in to a shared library written in C.  The shared library only needs to be loaded once, and is not accessible to scripts on web pages.  The library is just a vanilla C shared library, you don't have all the overhead of conforming to the NPAPI plugin interface.  The drawback, of course, is that you have to create both an NPAPI plugin and a js-ctypes shared library if you want to support Chrome and Firefox.  But I felt it was worth the extra effort, given the limitations of calling NPAPI plugins from Firefox extensions.

To minimize code duplication, I created a static library which contains most of my functionality, and then linked it into the NPAPI plugin and the js-ctypes shared library.

Dynamic bookmarks for developers

Some information here on the Eclipse plugin I have been working on:

Reverb Eclipse plugin on Google Project Hosting

The idea is to automatically recommend a set of web links based on the Java code you are currently working on in the IDE. To help ensure these are links you are interested in and may actually find useful, they are drawn from your own web browsing history. We will be starting a user study soon -- I hope you will give it a try!