Coordinating Different Kinds of User Interactions

User interaction in a GraphComponent is driven by so-called input modes. An input mode can be seen as a higher-order event handler which can reacts to sequences of events raised by the GraphComponent. Typically, these are mouse, touch, and keyboard events. For example, moving an item around requires first a “left mouse button down” event followed by a number of “drag” events with the left mouse button held down. The move gesture is successfully finished by a “left mouse button up” event or canceled by an “Escape key pressed” event. All these events are handled by the MoveInputMode internally. Therefore, you don’t have to care about them yourself — as long you don’t want to change the gestures.

The most common case is to use one of the larger composite input modes GraphEditorInputMode or GraphViewerInputMode. These input modes themselves delegate to a number of specialized input modes, their child modes, which themselves handle a single kind of user interactions. For example, GraphEditorInputMode delegates to its CreateEdgeInputMode for edge creation. They also coordinate the different child input modes to prevent them from interfering with each other. For example, the aforementioned “Mouse down — mouse drag — mouse up” gesture is used by MoveInputMode, MoveViewportInputMode, and MarqueeSelectionInputMode to move items, move the entire viewport, or draw a selection rectangle, respectively. It is the task of the higher order input modes to coordinate between these input modes to avoid that they interfere with each other.

Priorities and Exclusiveness

As mentioned above, some user interactions share the same gestures for user interaction. The “Mouse down — mouse drag — mouse up” gesture, e.g., can move the view port, move a node, or draw a selection rectangle. The higher order input modes GraphEditorInputMode and GraphViewerInputMode are so-called multiplexing input modes because they coordinate a number of input modes. In the following, let’s speak of the multiplexing input modes as parent modes of their child input modes.

Input modes may have phases during which no other input mode may interfere. This is handled via a mutex that input modes can request and release at certain points. This ensures that usually at the start of a specific gesture a single input mode can request to run exclusively until it releases the mutex again.

Once an input mode has the mutex from its parent mode all other child modes of the same parent are temporarily deactivated. They will not react to any events until the mutex is released again.

This also highlights another point: Input modes can usually only be activated after certain conditions have been met. For example, MoveInputMode only changes the mouse cursor when it is hovering over an item that can be moved. Most of the time, most input modes are just quietly observing and doing nothing, except changing the mouse cursor to indicate interaction possibilities.

Input modes have a priority that enables different input modes to have the same or similar activation gestures. For example, the press-drag-release gesture is used by MoveInputMode, CreateEdgeInputMode, MarqueeSelectionInputMode, and LassoSelectionInputMode. While MoveInputMode and CreateEdgeInputMode make sure to only perform actions when starting the drag on a selected or unselected element, respectively, e.g., MarqueeSelectionInputMode has no such provisions and would override the functionality of the other two when given a higher priority.

The priority property is reversed in its meaning: Higher priorities correspond to lower values of the priority property.

After an event (e.g. a mouse event) is raised/dispatched by the GraphComponent all active registered input modes receive that event in order of their priority.

The following example shows how to chose the priorities for different input modes with the same starting gesture (press-drag):

As a rule of thumb the input mode whose starting gesture is the most specialized one (MoveInputMode, mouse over selected nodes) has the highest priority, the most general one has the lowest priority. This way, the MarqueeSelectionInputMode, which can start anywhere on the canvas, will only be considered if the other input modes don’t have claimed to handle the gesture, yet.

The default priorities of the child modes on GraphEditorInputMode and GraphViewerInputMode have been chosen to go well with the default gestures. If one wants to customize the input, though, it might become necessary to customize the priorities, too.

For example, if one wants unselected nodes to be movable but also edge creation at the same time, both gestures now can start with a press and drag on nodes. You can limit the edge creation gesture to start only over a valid port candidate. However, the higher prioritized MoveInputMode will still always precede the CreateEdgeInputMode. We overcome this problem by following our rule of thumb: The CreateEdgeInputMode's now is higher specialized (hover over a port candidate of a node) and thus should get a higher priority than the MoveInputMode.

// Enable the move input mode for unselected items
graphEditorInputMode.moveUnselectedInputMode.enabled = true
// let CreateEdgeInputMode start over candidates only
graphEditorInputMode.createEdgeInputMode.startOverCandidateOnly = true
// let the MoveUnselectedInputMode have a lower priority than the CreateEdgeInputMode
graphEditorInputMode.moveUnselectedInputMode.priority = graphEditorInputMode.createEdgeInputMode.priority + 1

There are use cases where simply changing the priorities will not help. For example, marquee selection and panning both start with a drag on the canvas. If we want to use both we have to modify the starting gesture of one of them. This is described in the next section.

Event Recognizers

After an event (e.g. a mouse event) is raised by the GraphComponent all active registered input modes process this event and decide whether and how to handle that event.

For many of the input modes the events to which they react are configurable. They use EventRecognizers which can be set via property for this decision. An event recognizer is a function to which the event arguments are passed as parameter and which returns true if the event should trigger the action.

For example, the default gesture which starts both marquee selection and viewport panning is mouse down — mouse drag. Also, both are supposed to start on an empty part of the canvas. In order to use both of them we have to change the starting gesture of one of them. We can do this by setting the pressedRecognizer to react on Ctrl + Left Mouse Pressed:

graphEditorInputMode.marqueeSelectionInputMode.enabled = true
graphEditorInputMode.moveViewportInputMode.enabled = true
graphEditorInputMode.moveViewportInputMode.pressedRecognizer = EventRecognizers.createAndRecognizer(
  MouseEventRecognizers.LEFT_DOWN,
  KeyEventRecognizers.CTRL_IS_DOWN
)

Note that the above configuration is indeed the default in GraphEditorInputMode.

You might have noticed in the above example that predefined event recognizers are used. Predefined event recognizers for the most commonly used events available on KeyEventRecognizers, MouseEventRecognizers, and TouchEventRecognizers.

graphEditorInputMode.moveInputMode.pressedRecognizer = MouseEventRecognizers.LEFT_DOWN

Event recognizers can be easily combined

const moveInputMode = graphEditorInputMode.moveInputMode
moveInputMode.hoverRecognizer = EventRecognizers.createAndRecognizer(
  moveInputMode.hoverRecognizer,
  KeyEventRecognizers.SHIFT_IS_DOWN
)

For special purposes, custom event recognizers can be created:

const moveInputMode = graphEditorInputMode.moveInputMode
moveInputMode.hoverRecognizer = (source, arg) =>
  arg instanceof MouseEventArgs &&
  moveInputMode.hitTestable.isHit(graphEditorInputMode.inputModeContext, arg.location)

const moveInputMode = graphEditorInputMode.moveInputMode
moveInputMode.hoverRecognizer = (source: any, arg: EventArgs | null): boolean =>
  arg instanceof MouseEventArgs &&
  moveInputMode.hitTestable.isHit(graphEditorInputMode.inputModeContext!, arg.location)