Wednesday, May 18, 2011

AdaptableGIMP has been released!

Nearly two years after posting our first mockups on this blog, we're proud to announce that AdaptableGIMP has been officially released!

For more information, including downloads, check out

As always, we welcome comments, questions, and suggestions! Post them here or send us an email at

Monday, May 31, 2010

Updated Adaptable GIMP Mockups (May 2010)


In 2007, our lab released ingimp, a version of the open source GNU Image Manipulation Program (GIMP) that has been modified to gather rich usage data from its users. All gathered data is freely available at The idea was to give HCI researchers and software developers a picture of how this sophisticated software application was used in practice.

To date, we have gathered data from over a thousand users. Recently, we published our analysis of 2-years worth of ingimp data including logs from more than 200 users. We found that the typical ingimp user performs relatively short, targeted tasks with the application, using only a small fraction of the commands and tools available.

With this as motivation, we are building Adaptable GIMP, a modification of GIMP that adds an adaptable interface. That is, an interface that allows the user to easily simplify the application to fit the task that they are trying to accomplish.

In this document, we present our latest design mock-ups for Adaptable GIMP. These are the latest in a long line of evolutions of the previous set of mock-ups posted in May 2009. We welcome (and encourage) comments, questions, and suggestions. Help us make these designs better! (Please see for specific information on how we will use comments posted to this blog.)


Our adaptable interface to GIMP is based around task sets. Task sets are collections of tools and commands that are used to accomplish a given task. For instance, a task set for painting might include the Paintbrush, Pencil, and Eraser tools, but would probably not include the Sharpen tool. While the user was using a painting task set, only the tools included in the task set would be visible in the interface, reducing unneeded interface complexity. The notion of a task set is somewhat akin to perspectives in Eclipse---a way to organize one's workspace around a specific task.

Task sets may also include rich, user-generated documentation. This documentation may be used to describe a particular workflow with the task set, describe the rationale for including certain commands, or serve as a tutorial for performing a certain task. This is a second use for task sets: communicating how to use the application to accomplish tasks.

In our current design, task sets are created by users. When a user creates a task set, it is uploaded to our server and instantly made available to the entire user community. This openness makes it extremely easy for people to share interface adaptations and information on how to use them.

Given that introduction, let's jump into the mock-ups.

Overall Interface

When a user has installed and is using a task set, its tools and commands show up in a modified toolbox. The rich description document for the task set can also be displayed in a window within the application.

As mentioned above, once a task set has been created, it is shared with the entire community. At this point, both the commands and tools making up the task set as well as the description of the task set, are editable by any member of the community. When a user installs a task set, a copy of that task set is made on the users' system, including the commands and tools that were in the community task set at that time. As a result, that user's installation of the task set may become out of sync with the community version if changes are made. The user may choose to update to the latest version to follow changes made by the rest of the community.

In contrast to the commands and tools in a task set, when a viewer opens the description of a task set, the application always tries to fetch the most recent version. That is, the description document acts like a web page.

In fact, task sets and the description documents associated with them are accessible on a website associated with the project. The website provides a wiki-like interface for editing task set descriptions, and provides information on how the task set is being used by the user community.

Hopefully this has given a high-level overview of the interface. We'll now walk through a number of specific user interactions with the application.

Using Adaptable GIMP for the first time

The user starts by downloading and installing the application. When run for the first time, the user is presented with the toolbox, which asks for a username and password.

The user clicks create new user and a wizard pops up. Step 1 presents the user with information on the Adaptable GIMP project including a research consent form. Step 2, asks the user to enter a username, password, and password confirmation. The box for username has a “check availability” button beside it, which allows the user to find a username that is not yet being used. Step 3 informs the user: “Your account has been created”, and volunteers a brief tour of how Adaptable GIMP works. The user can choose to dismiss this tour and start using the application immediately instead. Once the wizard completes, the user is automatically logged in to their newly created account.

Navigating installed task sets

Selecting the active task set - User selects an installed task set from the drop down menu. The commands contained in this task set are then displayed in the toolbox, along with supplementary information about the task set.

Grid and Detail views - The user may select between two views of the active task set. In the Grid view (below left), commands are shown as icons without descriptions, arranged in a grid similar to how commands are typically arranged in the standard GIMP toolbox. In Detail view (below right), icons for commands are arranged vertically, with their names to their right, and an Info button. The info button for each command brings up a window describing the command. This window is similar to the description document window for task sets, and includes both information from GIMP's help files, as well as information created-by and editable-by the community. Each command has one description across all task sets (that is, the description document for a given command is the same, no matter which task set the user was in when they clicked the info button).

Forward and Back buttons - Navigation through task sets acts like navigation through recently-visited web pages. A back stack stores previously active task sets, and the Forward and Back buttons allow the user to navigate the stack. We decided upon this interaction for navigating installed task sets because it is well suited for working on tasks that have a hierarchical nature, including sub-tasks for which a user might want to use a different task set, before returning to the main task.

We would also like to include keyboard shortcuts for Forward and Back operations, to support fast switching.

Viewing a Description Document

Clicking the Info button for a task set (displayed in the Toolbox) brings up a window containing the description document for that task set. This window is non-modal, to allow the user to keep it open for reference while working. For the same reason, the window, by default, is thin and tall so that it may be placed unobtrusively beside the canvas on which the user is working.

When connected to the Internet, the application loads the current version of the task set's description document from the web. The extra information displayed around the description on the website can be hidden to conserve space. Below is an example of a task set description document that a hypothetical use has used to author a tutorial.

As mentioned above, the application tries to load the latest version of the description document. However, if the user is not connected to the Internet, then a cached version of the document from the last time the user was connected is displayed, along with a warning that the information may not be up to date, and instructions on how to view the latest version.

Installing new task sets

Searching for and installing task sets can be done right from the Toolbox. The user types keywords to search for into the search box. While typing, a window pops up listing task sets that match the entered search terms.

The user can use the arrow keys to navigate down and highlight one of the search results, and hit the [enter] key to install the task set. Alternately, the user can click the Info button to bring up the description document for a task set, or click the Add Task Set button to install the task set. When a task set is installed, it is also set as the current active task set.

In the case where a task set in the search results is already installed, selecting the task set will set the currently installed version of that task set as active.

We would also like to include an application-wide keyboard shortcut for moving focus to the search box. Ideally this could support an interaction where, while working, a user could switch task sets entirely through use of the keyboard. For instance, the user could hit a keyboard shortcut, bringing focus to the quick search box. They could then use the interaction described above to search and install a new task set, or select an installed task set. Once the task set was selected, focus would return to what the user was working on, allowing them to begin using the task set.

Adding or removing commands from task sets

There are two scenarios where a user might want to add or remove commands from a task set: 1) for personal use, or 2) to change the community version.

In the first case, the user wants to make small personal modifications to the installed version of a task set. Perhaps they want to add a command that they use frequently, but is not traditionally associated with the task set's task. Or they might want to add a command to invoke a plugin that is only available on their system.

To do this, the user first makes the task set that they wish to edit the active task set. They then click the Edit Commands button below the list of commands in the task set. This brings up the Task Set Editor dialog, shown below.

This dialog displays the current state of the task set on the right, and provides a range of ways for navigating the commands available in GIMP on the left. Commands from the left side can be added using the Add button. On the right, commands can be removed from the task set and reordered.

The second scenario is where the user wants to make changes to the community copy of a task set. In this case, the user first navigates to the task set’s description document page, either on the web or within the application. Once there, the user clicks an [Edit] link in the Commands section.

This would bring the user to a web-based, wiki-style interface for editing the commands in the task set. The intention of this separate interface for editing the community version of a task set's commands is to clearly differentiate between making changes that will only affect the user's local system, and changes that will affect other users in the community.

This community command editing functionality can also be accessed through a hyperlink from various places within the application, including the description document for the task set. These will launch a browser and load the web-based task set command editor.

Merging community changes into an installed task set

As mentioned above, when changes are made to the community version of a task set, they are not automatically pushed to users of that task set. That is, installing a task set installs a copy of the current version NOT a subscription. As well, as mentioned above, a user may have made small changes to their installation of a task set.

The user may want to 1) update an installed task set to the latest version, or 2) view and merge changes from the latest community version with the changes that they have made to their installed version.

Users are notified that the community version has deviated from their installed version from the description box for the active task set. If there are changes in the community version, the date field is displayed in green and italicized, and when the user passes their mouse over this area, a tooltip is displayed which reads “There are changes from your installed version. Click [here] to merge changes”.

To merge changes, the user clicks the link in the tooltip mentioned above. This will open the Task Set Editor dialog, with the “Most Recent Version” option selected in the area on the left.

Note that this is the same window loaded by clicking “Edit Commands” as described previously, and so the user could also see changes in the community version by loading this window using the "Edit Commands" button and then selecting the “Most Recent Version” option in the left hand side of the window.

In this view, the Task Set Editor dialog displays the commands in the current community version on the left. Subtle blue highlighting indicates commands that are in both the locally installed version and the current community version. This makes it easy for the user to see the changes that they have made to their local version, and changes in the community version relative to their local version. Additionally, there is also a Synchronize button displayed below the standard Add arrow button. When clicked, the Synchronize button overwrites any local changes, effectively installing the latest community version.

Creating a new task set

To create a new task set, the user starts by selecting "Create a new task set" from the task set selection drop down in the Toolbox. This loads a modified Task Set Editor window containing a blank task set.

The user enters a name for the task set, and adds the desired commands. On the right, below the commands box, there is a small WYSIWYG rich text editing area for adding an initial description for the task set. Note that, in the Task Set Editor windows that we have seen previously, this area merely displays the current state of the description document for the task set, and does not allow editing. All editing for created task sets is done through a web interface using wiki-syntax, as we describe in the next section.

The reason that this window does include simple WYSIWYG functionality is to provide a gentle introduction to wiki-syntax for users who may be unfamiliar it, and for whom this might be a barrier to creating a task set. Users can use this simple editing functionality to provide an initial description, and see the wiki-syntax equivalent when they go to make further edits through the web interface, scaffolding their learning of wiki-syntax.

Editing a task set's description document

As mentioned above, when the task set is first created the user can enter a basic description using WYSIWYG tools. Further editing of a task set’s description is done by visiting the task set's description document on the web, and clicking [Edit]. This brings the user to a wiki-syntax editing interface where they can make further changes, and upload attachments such as photos.

That's all for now!

Hopefully that covers the most important interactions that we have designed for Adaptable GIMP, and gives a flavour of what our current plans are for the system as a working whole.

As mentioned previously, these designs are constantly evolving, and we value any and all feedback you can provide.

Thanks a lot!

-- Ben Lafreniere (May 31, 2010)


B. Lafreniere, A. Bunt, J. Whissell, C. L. A. Clarke, and M. Terry. Characterizing Large-Scale Use of a Direct Manipulation Application in the Wild, Proceedings of GI 2010. 8 pages. [PDF]

Tuesday, May 26, 2009

Adaptable GIMP Mockups: May 26, 2009

The ingimp dataset indicates that people only use about 12 different commands when they use GIMP. Given that there are over 300 commands in the interface, this finding suggests that we can significantly simplify the interface to only show those commands necessary to complete a task.

The HCI Lab at the University of Waterloo is designing a new version of GIMP, Adaptable GIMP, that incorporates an adaptable interface, or an interface that can modify its presentation depending on a user’s needs and tasks.

This blog contains a set of mock-ups of this potential redesign. We welcome comments, questions, and suggestions for improvements! (Please see for specific information on how we will use comments posted to this blog.)

Simplification Through Task Sets

To simplify the interface, we introduce task sets, or collections of tools and commands typically used to accomplish a given task. For example, when painting, one typically uses the paintbrush, pencil, and eraser tools, but is less likely to employ Unsharp Mask. Thus, a painting task set would include tools common to painting, and only show these tools in the interface. The notion of a task set is somewhat akin to perspectives in Eclipse -- a way to organize one's workspace to cater to a specific task.

Note: It is important to distinguish between tools and commands. Tools are instruments used to directly interact with an image, and include things like the paintbrush or smudge tool. Commands, on the other hand, operate on regions of pixels at once. We sometimes use these terms interchangeably, though at times it is important to distinguish between the two when considering the interaction design for Adaptable GIMP. To reiterate, a task set includes a select set of both commands and tools.

Creating and Sharing Task Sets

In our current design, task sets are manually created by users. Any task set a user creates is uploaded to our server where it is instantly made available to the entire community. Thus, this adaptable interface is open, making it extremely easy for people to share adaptations, customizations, and even tutorials (more on that later).

Given this overview, let’s jump right into the mock-ups.


Note: The mock-ups are intentionally rough in their appearance and layout. We are primarily concerned with functionality, features, and workflow at this point, that is, ensuring we have all the functionality required for an initial release. That said, we welcome specific suggestions on how to best present the functionality described.

Toolbox: Grid View

In Adaptable GIMP, the GIMP toolbox is modified to include:
  • a drop-down menu of task sets,
  • a set of additional tool views, and
  • a button to edit the current task set.
The task set drop-down menu contains the following items:
  • the application defaults (i.e., all tools)
  • --
  • the user’s installed task sets
  • --
  • the task sets used most frequently by the entire community
  • --
  • an option to create a new task set
The additional tool views toggle presentation of the contents of the Toolbox between:
  • Icons with no text
  • Icons with text
  • Icons with text and additional information

Toolbox Contents

In Adaptable GIMP, we allow menu commands to be shown in the Toolbox. Thus, all commands and tools in a task set are visible in one place, namely, the Toolbox. We believe this will significantly simplify the interface by providing the user a single place to search for tools and commands. Given the low number of operations performed by a user in any given session, this should be feasible.

The “Edit Task Set...” button opens up a modal dialog box (described below) for editing the task set.

Toolbox: List View

The List View of the Toolbox shows each tool/command with its natural language name. We believe this will help novice users of the application.

Toolbox: Information View

The informational view allows the user to view notes for a given tool/command or the current task set. Users select the “i” icon next to the tool they wish to get more information about, or the “i” icon next to the task set name (top-right) to obtain information about the task set. The current design is a bit busy; we may remove the color around the "i" icons to make it a bit cleaner.

In the current design, the user has to click an "i" icon. Some may ask why this is (e.g., why not just click the tool/command of interest to get more info?). The reason for this design is that both tools and commands may appear in the toolbox: Since commands are invoked immediately when clicked, we need to provide a way for users to receive information about a command without invoking the command itself.

Information for a tool can be edited by clicking the “Edit Tool Info...” button. Any changes made to the text will be uploaded to the server and made available instantly to all Adaptable GIMP users. In this way, the tool info is somewhat like a wiki page. Because changes will be made available to everyone, the user will be given a reminder that any changes are global in effect.

If the user clicks the “i” icon next to the task set, the task set’s information is displayed and the edit button’s text is changed to read “Edit Task Info...” Editing the task set’s info is done in the task set editor window, discussed below.

The task set info can be used to communicate information to assist in using the task set. For example, the content may comprise a tutorial instructing users how to use the chosen tools to accomplish a specific task.

Task Set Manager

Clicking the wrench icon in the Toolbox (next to the task set drop-down menu in the Toolbox) invokes the Task Set Manager (click the image above to get a larger version of the screenshot). This manager allows the user to install/uninstall task sets in the drop-down menu.

By default, the Task Set Manager opens to the screen above. Users can toggle between showing available task sets and those which are already installed. Users can search for task sets and sort via the following options:
  • Most popular
  • Recently used by the community
  • Recently added
  • Alphabetical
Selecting a task set in the list of task sets displays more information about the task set as well as a preview of the commands. Users can install selected task sets via the install button and immediately switch to the last selected task set via the “Use This Task Set Now” button.

Toggling the “Installed” button shows the task sets currently installed. Users can uninstall selected task sets, reorder the order in which they are presented in the drop-down menu in the Toolbox (by using the arrow buttons), and can edit the task set by clicking the “Edit...” button.

Task Set Editor

Clicking the “Edit Task Set” button on the Toolbox brings up the Task Set Editor. There are two copies of any task set: The user’s personal copy and the community copy. When installing a task set, a copy of the community copy is made on the user’s machine. This copy does not change unless the user manually makes modifications to it. This behavior is to prevent the task set changing unexpectedly on the user should someone edit the community copy of the task set.

Any changes made to the personal copy are only made locally. Users can only add/remove commands from the personal copy; no edits can be made to the local description of the task set or its name. The order of the commands/tools (as they appear in the Toolbox) can be modified using the arrow buttons.

Users can add commands to a task set by selecting them in the list on the right. Users can filter the set of possible commands to add by choosing:
  • All commands
  • Menu items
  • Toolbox items
  • Frequently used with this Task Set
  • Frequently used by community
  • Frequently used by you

Choosing the “Community Copy” tab allows a user to edit the “master” copy of a task set. This is the copy of the task set users receive when installing a task set (any changes are not retroactively applied after the task set has been installed). Anyone can edit this version of the task set.

Users can edit the tags describing this task set, its description, and the set of commands included with the task set. Users can also change the locale for the task set to localize text. Finally, users can view past versions of a task set, reverting to a prior version, if desired.

Version Control

Past versions of the task set can be accessed in the dialog box shown above. Versions are listed on the left, the tags, description, and commands in that version are shown on the right. Users can synchronize either their personal copy or community copy to a selected version.

Next Steps

If you made it this far, congrats, that was a lot of text :)

From here, please post any feedback, comments, or suggestions below in the Comments section.