Table of Contents
- VGG Image Annotator (VIA)
- VIA Buttons-for-Labels Modification
- VIA version 2.0.0
* VIA Attribution and Citation
You got your great new machine learning computer setup. Lots of cores, lots of memory, an NVIDIA 1080Ti or two, maybe a nice Titan V. Linux is all setup, Anaconda Python, TensorFlow, Keras, installed. And, you have a bunch of fresh image data and a “real” problem to solve! Now what?
If it’s newly generated real world image data then there is a good chance that you need to do some object annotation to get your training data in a usable state. How are you going to do that? Maybe you’ll,
- Pay random people on Amazon’s Mechanical Turk to do it for you using one of the available annotation packages available there.
- Get your slave labor, i.e. graduate students, to spend 16 hours a day working on it.
- Do it yourself (hopefully with the help of friends, colleagues, or co-workers).
I don’t have graduate students and the data for one of my projects is private so it can’t be “crowd sourced” or transferred to any on-line service. But, I do have willing co-workers (I hope). It’s just a matter of getting a good, simple to use, annotation tool working that I can make available to my co-workers.
It turns out that there are lots of image annotation tools available but almost none of them meet my needs! Just search on-line for image annotation tools or look at the list on Wikipedia.
My annotation tool requirements are,
- Easy to use
- Fast (I can’t have people spending tens of hours working on this)
- It can’t require any software install with dependencies like Python or Java etc..
- It needs to be portable from work to home and not require a web server.
- It should be free (since there is no official budget)
Usage,
- I need to generate simple bounding box coordinates with labels
- I only need to label a few classes of objects
- Did I mention fast and easy …
It turned out that it was difficult finding any annotation tool that could meet those requirements. I did find one that was “almost great”. This post is about truning that “almost great” into “great” (for my use case).
VGG Image Annotator (VIA)
VIA is a very useful open-source image annotator. It’s one of many interesting projects under active development at the University of Oxford Visual Geometry Group (VGG).
Some of the nice features of VIA are,
- Entirely HTML, CSS and Javascript (no external javascript libraries)
- It’s contained in one html file. That file can be opened in a browser and used of-line.
- It has a variety of region shapes (I only need rectangles)
- …
VIA is fairly easy to use and mostly intuitive. However, in version 1.0.x it is slow to use. You can easily traverse a list of images and simply use the mouse to draw bonding boxes, but, you have to enter region labels by typing them in a spreadsheet like interface in a different part of the GUI.
Here’s an example of that,
The simple mod that I’m going to make will add easy to use buttons placed under the image. It is very fast to use!
That hopefully that screen-shot above gives you an idea of how fast and easy this can be to use. You can keep one hand on the mouse for drawing the bounding boxes and clicking the label buttons, and the other hand on the right arrow key to change to the next image. In the next section I’ll show you a simple mod to the VIA source to add label buttons.
With this simple modification I found I could do bounding-boxs and labels with 3 classes at a rate of 15-20 images per minute.
VIA Buttons-for-Labels Modification
The modification that I describe here is for VIA version 1.0.6 released on June 15, 2018.
VIA 1.0.6 is contained in a single html file that does not load any external javascript libraries. It is nearly 5000 lines of mostly javascript. It did take me awhile to figure out how add the functionality that I wanted. What I did is not elegant or representative of good coding practice. It’s just a simple quick “hack” to make it do what I want.
There are two things I added,
- A set of html buttons right after the “div” for the image pane.
- A javascript function that the buttons call. This adds the bounding-box coordinates and annotations to the annotations file and updates the GUI state.
Adding the Buttons
I’ll use the code from the mod I used for the image of the man on the bicycle example. You can add more or fewer buttons and whatever labels you need.
Starting at line 746
<!-- // dbk mod -- add some label buttons under the image canvas -->
<input id="Hat" type="button" value="hat" onclick="label_image_region(this.value);" />
<input id="Bicycle" type="button" value="bicycle" onclick="label_image_region(this.value);" />
<input id="Person" type="button" value="person" onclick="label_image_region(this.value);" />
To customize this for your needs change the “id=” to the button label you want and the “value=” to the bounding-box annotation label you need.
Adding the javascript for “label_image_region()”
Starting at line 4590
// dbk mod -- function for button onclick, adds the image region label
function label_image_region(value) {
var attribute_name = 'label';
var region_id = _via_user_sel_region_id;
// write out annotation meta-data and update state
_via_img_metadata[_via_image_id].regions[region_id].region_attributes[attribute_name] = value;
update_region_attributes_input_panel();
if (_via_is_reg_attr_panel_visible) {
set_region_select_state(region_id, false);
}
_via_redraw_reg_canvas();
_via_is_user_updating_attribute_value = false;
save_current_data_to_browser_cache();
}
The Output Meta-Data
What you get is a file with meta-data for the image including bounding-box coords and labels. You can save the file as CVS or JSON. It looks like this,
#filename,file_size,file_attributes,region_count,region_id,region_shape_attributes,region_attributes
595px-Man_on_bicycle.jpg,52630,"{}",3,0,"{""name"":""rect"",""x"":287,""y"":65,""width"":84,""height"":42}","{""label"":""hat""}"
595px-Man_on_bicycle.jpg,52630,"{}",3,1,"{""name"":""rect"",""x"":281,""y"":78,""width"":152,""height"":335}","{""label"":""person""}"
595px-Man_on_bicycle.jpg,52630,"{}",3,2,"{""name"":""rect"",""x"":119,""y"":230,""width"":390,""height"":236}","{""label"":""bicycle""}"
The output file should be reasonably easy to parse to whatever format you need for the machine learning training algorithms you plan to use.
VIA version 2.0.0
The 2.0.0 version of VIA was released at the same time as the 1.0.6 update that I used for my mod. Version 2 is a major update. It adds new functionality including a much improved annotation editor. This version has the capability of adding “radio buttons”, “check boxes”, “drop down menus”, and “images” to use for adding annotations. I tried it and thought it was a big improvement over version 1 and I recommend you check it out. I was considering using it form my projects but even with the improved interface it is still not as quick and easy to use as my mod for version 1. However, for a projects that have more demanding annotation requirements it could be the best option available.
I did try to make my button mod to version 2 but after an entire day of digging through the code and debugging I gave up. If I had more time it would be a great project to spend the time with to understand and contribute to.
Remember, VIA just runs as a single file that runs in your browser so if you want to try out version 1 or version 2 just go to the VGG VIA site where you will find some live demos to try.
I hope this post helps you with your projects! The VIA version 1 mod is nearly perfect for what I’m working on.
I’ll end with a thank you to the VGG group for all of the great software they are making available to the community.
Happy computing –dbk
VIA Attribution and Citation
“Development and maintenance of VGG Image Annotator (VIA) is supported by EPSRC programme grant Seebibyte: Visual Search for the Era of Big Data (EP/M013774/1)”
@misc{ dutta2016via,
author = "Dutta, A. and Gupta, A. and Zissermann, A.",
title = "{VGG} Image Annotator ({VIA})",
year = "2016",
howpublished = "http://www.robots.ox.ac.uk/~vgg/software/via/",
note = "Version: 1.0.6, Accessed: June 27, 2018"
}