Coordinating Different Kinds of User Interactions

User interaction in a GraphComponent is driven by input modes. An input mode can be seen as a higher-order event handler that reacts to sequences of events raised by the GraphComponent. Typically, these are mouse, touch, and keyboard events. For example, moving an item requires a “left mouse button down” event, followed by a number of “drag” events while the left mouse button is held down. The move gesture finishes with a “left mouse button up” event or is canceled by an “Escape key pressed” event. All these events are handled internally by the MoveInputMode. Therefore, you don’t have to handle them yourself, unless you want to change the gestures.

The most common approach is to use one of the composite input modes GraphEditorInputMode or GraphViewerInputMode. These input modes delegate to a number of specialized input modes, their [def]#child modes, which handle a single kind of user interaction. For example, GraphEditorInputMode delegates to its CreateEdgeInputMode for edge creation. They also coordinate the child input modes to prevent them from interfering with each other. For example, the “Mouse down — mouse drag — mouse up” gesture is used by MoveInputMode, MoveViewportInputMode, CreateEdgeInputMode, and MarqueeSelectionInputMode to move items, move the entire viewport, create edges, or draw a selection rectangle, respectively. The higher-order input modes coordinate between these input modes to avoid interference.

Priorities and Exclusiveness

As mentioned above, some user interactions share the same gestures for user interaction. The “Mouse down — mouse drag — mouse up” gesture, for example, can move the viewport, move a node, or draw a selection rectangle. The higher-order input modes GraphEditorInputMode and GraphViewerInputMode are so-called multiplexing input modes because they coordinate a number of input modes. In the following, let’s refer to the multiplexing input modes as parent modes of their child input modes.

Input modes have phases during which no other input mode may interfere. This is handled via a mutex that input modes can request and release at certain points. This ensures that usually at the start of a specific gesture, only one input mode can request to run exclusively until it releases the mutex again. Once an input mode has obtained the mutex from its parent mode, all other child modes of the same parent are temporarily deactivated and will not react to any events until the mutex is released again.

Most of the time, input modes are just quietly observing and doing nothing. However, they may change the mouse cursor to indicate interaction possibilities. For instance, MoveInputMode changes the cursor when the mouse hovers over an item that can be moved. This also highlights an important point: input modes can usually only be activated after meeting specific criteria. In this example, MoveInputMode will only become active if the mouse is over a movable item.

Input modes have a priority that allows different input modes to have the same or similar activation gestures. When an event (e.g., a mouse event) is raised/dispatched by the GraphComponent, all active registered input modes receive that event in order of their priority. By utilizing different priorities, the input mode with the highest priority can take the mutex and, in turn, disable all other input modes.

The priority property is reversed in its meaning: Higher priorities correspond to lower values of the priority property.

As a demonstration, let’s look at input modes that share the same gesture. For example, with GraphEditorInputMode, the press-drag-release gesture is used by moveUnselectedItemsInputMode, moveSelectedItemsInputMode, createEdgeInputMode, marqueeSelectionInputMode, and lassoSelectionInputMode. While MoveInputMode and CreateEdgeInputMode make sure to only perform actions when starting the drag on a movable element or on a port candidate, respectively, the MarqueeSelectionInputMode has no such provisions and would override the functionality of the other two when given a higher priority. The following table shows how to choose the priorities for these input modes:

As a general rule, the input mode with the most specialized starting gesture (CreateEdgeInputMode, mouse over port candidate) is given the highest priority, while the most general one is assigned the lowest priority. For this reason, the MarqueeSelectionInputMode, which allows starting anywhere on the canvas, is considered only if other input modes have not claimed to handle the gesture.

The default priorities of the child modes on GraphEditorInputMode and GraphViewerInputMode have been chosen to work well with the default gestures. If one wants to customize the input, though, it might become necessary to customize the priorities, too.

To illustrate this principle, let’s discuss how these different modes are orchestrated by default in GraphEditorInputMode: moveUnselectedItemsInputMode starts with a drag over a node. The less specific moveViewportInputMode, lassoSelectionInputMode, and marqueeSelectionInputMode which can start anywhere on the canvas need a lower priority. Therefore, the moveUnselectedItemsInputMode will receive the mutex before them. On the other hand, the moveSelectedItemsInputMode will start over selected nodes only and, therefore, is more specific than moveUnselectedItemsInputMode. Finally, the CreateEdgeInputMode class by default starts over a port owner, i.e., an unselected node. To make it more specific, GraphEditorInputMode configures it by default to start only by dragging from a port candidate.

The child modes moveViewportInputMode, lassoSelectionInputMode, and marqueeSelectionInputMode still require another distinguishing feature. GraphEditorInputMode configures moveViewportInputMode to start only when the Ctrl key is pressed. Setting specific event recognizers is discussed further in the next section. Finally, lassoSelectionInputMode is disabled by default. With GraphEditorInputMode's default configuration, it is not possible to use marqueeSelectionInputMode and lassoSelectionInputMode at the same time.

In the following example, we are changing the behavior so that only selected nodes can be moved. Additionally, edge creation can start over the entire node, not just the port candidate (as per yEd’s default configuration). To achieve this, we first disable moveUnselectedItemsInputMode. Then, we configure createEdgeInputMode to start from any point over a node. Finally, we have to set the priority of the higher specific input mode moveSelectedItemsInputMode (selected nodes) higher than the less specific input mode createEdgeInputMode (unselected nodes).

// disable moving unselected items
graphEditorInputMode.moveUnselectedItemsInputMode.enabled = false
// switch priorities of MoveSelectedItemsInputMode and CreateEdgeInputMode
graphEditorInputMode.moveSelectedItemsInputMode.priority = 100
graphEditorInputMode.createEdgeInputMode.priority = 110
// let edge creation start over the entire node
graphEditorInputMode.createEdgeInputMode.startOverCandidateOnly = false
// don't show start port candidates
graphEditorInputMode.createEdgeInputMode.showPortCandidates =
  ShowPortCandidates.END

There are use cases where simply changing the priorities will not help. As mentioned above, marquee selection and panning both start with a drag on the canvas. If we want to use both, we have to modify the starting gesture of one of them. This is described in the next section.

Event Recognizers

After an event (e.g., a mouse event) is raised by the GraphComponent, all active, registered input modes process this event and decide whether and how to handle it.

For many input modes, the events to which they react are configurable. They use event recognizers, which can be set via property to determine this. An event recognizer is a function that accepts the event arguments as a parameter and returns true if the event should trigger the action.

For example, the default gesture that starts both marquee selection and viewport panning is a mouse down followed by a mouse drag. Also, both are supposed to start on an empty part of the canvas. To use both, we must change the starting gesture of one of them. We can do this by setting the beginRecognizer to react to Ctrl + Left Mouse Pressed:

graphEditorInputMode.marqueeSelectionInputMode.enabled = true
graphEditorInputMode.moveViewportInputMode.enabled = true
graphEditorInputMode.moveViewportInputMode.beginRecognizer = (
  eventSource,
  evt
) =>
  EventRecognizers.MOUSE_LEFT_DOWN(eventSource, evt) &&
  EventRecognizers.CTRL_IS_DOWN(eventSource, evt)

Note that the above configuration is similar to the default in GraphEditorInputMode, which additionally reacts to Space + Left Mouse Pressed and to Middle Mouse Pressed.

You might have noticed in the above example that predefined event recognizers are used. Predefined event recognizers for the most commonly used events are available in EventRecognizers.

graphEditorInputMode.moveSelectedItemsInputMode.beginRecognizer =
  EventRecognizers.MOUSE_LEFT_DOWN

Event recognizers can be easily combined.

const moveInputMode = graphEditorInputMode.moveSelectedItemsInputMode
moveInputMode.hoverRecognizer = (eventSource, evt) =>
  moveInputMode.hoverRecognizer(eventSource, evt) &&
  EventRecognizers.SHIFT_IS_DOWN(eventSource, evt)

For special purposes, custom event recognizers can be created:

const moveInputMode = graphEditorInputMode.moveSelectedItemsInputMode
moveInputMode.hoverRecognizer = (arg) =>
  arg instanceof PointerEventArgs &&
  moveInputMode.hitTestable.isHit(
    graphEditorInputMode.createInputModeContext(),
    arg.location
  )

const moveInputMode = graphEditorInputMode.moveSelectedItemsInputMode
moveInputMode.hoverRecognizer = (arg: EventArgs | null): boolean =>
  arg instanceof PointerEventArgs &&
  moveInputMode.hitTestable.isHit(
    graphEditorInputMode.createInputModeContext(),
    arg.location
  )