Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add XR support to the cursor so that it can be used with XR select ev…
…ents
  • Loading branch information
AdaRoseCannon committed Jul 7, 2022
commit 2a6d937a0fa8b873cef126886f8be4cda418734e
15 changes: 14 additions & 1 deletion docs/components/cursor.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ AFRAME.registerComponent('cursor-listener', {
| fuse | Whether cursor is fuse-based. | false on desktop, true on mobile |
| fuseTimeout | How long to wait (in milliseconds) before triggering a fuse-based click event. | 1500 |
| mouseCursorStylesEnabled | Whether to show pointer cursor in `rayOrigin: mouse` mode when hovering over entity. | true |
| rayOrigin | Where the intersection ray is cast from (i.e.,entity or mouse). `rayOrigin: mouse` is extremely useful for VR development on a mouse and keyboard. | entity
| rayOrigin | Where the intersection ray is cast from (i.e. xrselect ,entity or mouse). `rayOrigin: mouse` is extremely useful for VR development on a mouse and keyboard. | entity
| upEvents | Array of additional events on the entity to *listen* to for triggering `mouseup` (e.g., `trackpadup` for daydream-controls). | [] |

To further customize the cursor component, we configure the cursor's dependency
Expand Down Expand Up @@ -183,3 +183,16 @@ pick up event with the `begin` attribute:

To play with an example of a cursor with visual feedback, check out the [Cursor
with Visual Feedback example on CodePen][cursor-codepen].

## XR Select Cursor

When an XR `"selectstart"` event happens the raycaster picks an element based upon it's current location.
This works for handheld AR, and headmounted VR and AR. This works well with the mouse `rayOrigin` too.

```html
<a-scene
cursor__mouse="rayOrigin: mouse"
Copy link
Member

@dmarcos dmarcos Jul 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiple components for this might feel like an overkill. wondering if we can simplify somehow. maybe rayOrigin: mouse also considers selectstart? Not a fan either because makes rayOrigin: mouse a bit deceiving (not really only about mouse anymore)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can think of a few situations in which a developer may prefer to handle all of their own XR select behaviours but still take advantage of the mouse cursor.

My XR Input Handling library in particular makes it really simple to start building your x-platform own XR cursor using the entity method of the cursor component component.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any other ideas to avoid the multiple component approach?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have rayOrigin be an array of strings?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

array would weird too I think. probably even weirder. Let's go with the multiple components for now but I don't feel super comfortable. Thanks for the patience

cursor__xrselect="rayOrigin: xrselect"
raycaster="objects:#objects *;"
>
```
42 changes: 38 additions & 4 deletions src/components/cursor.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,11 @@ module.exports.Component = registerComponent('cursor', {
fuseTimeout: {default: 1500, min: 0},
mouseCursorStylesEnabled: {default: true},
upEvents: {default: []},
rayOrigin: {default: 'entity', oneOf: ['mouse', 'entity']}
rayOrigin: {default: 'entity', oneOf: ['mouse', 'entity', 'xrselect']}
},

multiple: true,

init: function () {
var self = this;

Expand Down Expand Up @@ -223,7 +225,16 @@ module.exports.Component = registerComponent('cursor', {
mouse.x = (left / bounds.width) * 2 - 1;
mouse.y = -(top / bounds.height) * 2 + 1;

if (camera && camera.isPerspectiveCamera) {
if (this.data.rayOrigin === 'xrselect' && evt.type === 'selectstart') {
const frame = evt.frame;
const inputSource = evt.inputSource;
const referenceSpace = this.el.renderer.xr.getReferenceSpace();
const pose = frame.getPose(inputSource.targetRaySpace, referenceSpace);
const transform = pose.transform;
direction.set(0, 0, -1);
direction.applyQuaternion(transform.orientation);
origin.copy(transform.position);
} else if (camera && camera.isPerspectiveCamera) {
origin.setFromMatrixPosition(camera.matrixWorld);
direction.set(mouse.x, mouse.y, 0.5).unproject(camera).sub(origin).normalize();
} else if (camera && camera.isOrthographicCamera) {
Expand All @@ -250,6 +261,24 @@ module.exports.Component = registerComponent('cursor', {
evt.preventDefault();
}

if (this.data.rayOrigin === 'xrselect' && evt.type === 'selectstart') {
Copy link
Member

@dmarcos dmarcos Jul 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this if be include in the if above to keep logic together

  if (evt.type === 'selectstart') {
     this.activeXRInput = evt.inputSource;
      this.onMouseMove(evt);
      this.el.components.raycaster.checkIntersections();
      ....
  } 

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disregard. github was not displaying enough context. sorry

this.onMouseMove(evt);
this.el.components.raycaster.checkIntersections();

// if something was tapped on don't do ar-hit-test things
if (
this.el.components.raycaster.intersectedEls.length &&
this.el.sceneEl.components['ar-hit-test'] !== undefined &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the component ar-hit-test be set on init if rayOrigin xrselect? For this to work it depends on the user to also set ar-hit-test separately

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not so much a dependency just that things can get messy if both are being used so if the user is tapping on a 3d object then it shouldn't also interact with the real world behind it

this.el.sceneEl.components['ar-hit-test'].data.enabled
) {
// Cancel the ar-hit-test behaviours and disable the ar-hit-test
this.el.sceneEl.components['ar-hit-test'].data.enabled = false;
this.el.sceneEl.components['ar-hit-test'].hitTest = null;
this.el.sceneEl.components['ar-hit-test'].bboxMesh.visible = false;
this.reenableARHitTest = true;
}
}

this.twoWayEmit(EVENTS.MOUSEDOWN);
this.cursorDownEl = this.intersectedEl;
},
Expand All @@ -269,14 +298,19 @@ module.exports.Component = registerComponent('cursor', {
var data = this.data;
this.twoWayEmit(EVENTS.MOUSEUP);

if (this.reenableARHitTest === true) {
this.el.sceneEl.components['ar-hit-test'].data.enabled = true;
this.reenableARHitTest = undefined;
}

// If intersected entity has changed since the cursorDown, still emit mouseUp on the
// previously cursorUp entity.
if (this.cursorDownEl && this.cursorDownEl !== this.intersectedEl) {
this.intersectedEventDetail.intersection = null;
this.cursorDownEl.emit(EVENTS.MOUSEUP, this.intersectedEventDetail);
}

if ((!data.fuse || data.rayOrigin === 'mouse') &&
if ((!data.fuse || data.rayOrigin === 'mouse' || data.rayOrigin === 'xrselect') &&
this.intersectedEl && this.cursorDownEl === this.intersectedEl) {
this.twoWayEmit(EVENTS.CLICK);
}
Expand Down Expand Up @@ -363,7 +397,7 @@ module.exports.Component = registerComponent('cursor', {
}

// Begin fuse if necessary.
if (data.fuseTimeout === 0 || !data.fuse) { return; }
if (data.fuseTimeout === 0 || !data.fuse || data.rayOrigin === 'xrselect') { return; }
cursorEl.addState(STATES.FUSING);
this.twoWayEmit(EVENTS.FUSING);
this.fuseTimeout = setTimeout(function fuse () {
Expand Down