Hand tracker: Calibration and occlusion

Before using the hand tracker, we should calibrate it. We'll also occlude AR assets that appear behind the hand.

Calibration

The hand tracker can sometimes project the hand landmarks into the wrong depth range. A calibration process is required to fix this error. It's a short process where we ask the user to place their hand on a flat surface, direct the camera on it and bring it closer and further for a few seconds while a plane ray-casting is performed to determine the distance from the camera to the plane on which the user has laid the hand. In the demo scene from the sample provided with the Ur package, you can find a Calibrate Hand Tracker Button and code that can easily be used in any project.

You will also need to add the ^ARPlaneManager^ component on the ^ARSessionOrigion^ GameObject.

Occlusion Culling

Add the ^AROcclusionManager^ component to ^AR Camera^ GameObject and adjust its properties (Fastest on all).

To use the ^AROcclusionManager^ in the code, import ^ARSubsystems^.

using UnityEngine.XR.ARSubsystems;

Declare an ^AROcclusionManager^ serializable variable and a boolean for toggling it on and off.

[SerializeField] private AROcclusionManager arOcclusionManager;
private bool occlusionBool = true;

In Unity, drag the ^ARCamera^ GameObject to the ^AROcclusionManager^ field in the ^ConureKitManager^.

Then create a toggle method to toggle between the different ^AROcclusionManager^ modes.

public void ToggleOcclusion()
{
    occlusionBool = !occlusionBool;

    arOcclusionManager.requestedHumanDepthMode = occlusionBool ? HumanSegmentationDepthMode.Fastest : HumanSegmentationDepthMode.Disabled;
    arOcclusionManager.requestedHumanStencilMode = occlusionBool ? HumanSegmentationStencilMode.Fastest : HumanSegmentationStencilMode.Disabled;
    arOcclusionManager.requestedEnvironmentDepthMode = occlusionBool ? EnvironmentDepthMode.Fastest : EnvironmentDepthMode.Disabled;
}

Use a UI toggle to switch between modes, but again you can use any method you choose.

Need a refresher on the essentials?

Check out more lessons, DIY kits and essentials reading material at the developer learning centre homepage.

Want to build on the posemesh and need a helping hand?

Apply for a grant of AUKI tokens to get your project off the ground, and work directly with the Auki Labs team to get your creation to market. Successful applicants may be granted up to 100k USD worth of AUKI tokens, and development and marketing support from the Auki Labs team.