Build a Vive / Tilt Brush Style Controller in Daydream

The differentiating feature of Daydream from other head locked VR systems on the market is its handheld controller. As a developer, there are a lot of cool tricks you can do with the controller to create a delightful experience and help your app really stand out. There’s the superhero effect: presenting wrist locked UI with the flick of a wrist. The HTC Vive effect: tracking the position of the user’s thumb as they move it around the touchpad. And the Tilt Brush effect: interacting with complex rotating wrist locked UI. In this tutorial, we’re going to do a couple of these things. First, we’ll create a simple mechanic that displays a dot that follows the x,y position of the user’s thumb and then we display a more complex three-dimensional UI system that can be swipe rotated.

Update (01/08/17): This tutorial was originally written using a pre-release version of the Daydream Unity SDK. The code in the tut is still totally valid, but the linked Unity package uses an older SDK so may not run in recent versions of Unity. See my tutorials here on setting up Daydream and here on building a working controller based app with teleportation that uses the release SDK.

Download the finished Unity package from here and import it into a new empty project. If you look at the assets you’ll see I’ve spent the whole of about 30 seconds creating a crappy looking controller. I expect you to try a bit harder than me in the art department. Take a look at the ControllerManagerScript to get an idea of how the code works. Interacting with the Daydream Controller’s touchpad is relatively simple. We communicate with it through the Controller API (a singleton class) and simply listen to its event calls from the Update loop. When the GVRController.isTouching is true we can access a TouchPosition (line 35) of a Vector2 with x,y between 0 and 1. Having a number between 0 and 1 isn’t very useful we need to map this to an actual localPostion on the gameObject so we can move a dot around the surface. So at lines 36 to 39 we map this Vector 2 to a percentage of the actual size of the thumb model and work out the position that way. The swipe code works by testing for a swipe (line 42), we see if a certain distance has been covered on the x-axis since the last update, if so I’m using DoTween library to rotate the UI cube by 90 degrees (line 45) in the direction of the swipe.

using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using DG.Tweening;

public class ControllerManagerScript : MonoBehaviour {

	public GameObject thumbTouchObj;
	public GameObject uiContainer;
	public GameObject trackPad;

	private Vector3 initTrackPos;
	private Vector3 trackPadSize;
	private float onePercentTrackPadSize;
	private Vector3 previousThumbPos;
	private bool isAnimating;

	void Start () {
		isAnimating = false;
		initTrackPos = trackPad.transform.localPosition;
		trackPadSize = trackPad.GetComponent<Renderer>().bounds.size;
		onePercentTrackPadSize = trackPadSize.x;
		thumbTouchObj.SetActive (false);

	void Update () {
		Quaternion ori = GvrController.Orientation;
		gameObject.transform.localRotation = ori;
		Vector3 v = GvrController.Orientation * Vector3.forward;

		if (GvrController.IsTouching) {
			// Track the thumb pos and move the dot
			thumbTouchObj.SetActive (true);
			Vector2 touchPos = GvrController.TouchPos;
			float xPos = (touchPos.x * onePercentTrackPadSize) + (initTrackPos.x - (onePercentTrackPadSize / 2.0f));
			float yPos = thumbTouchObj.transform.localPosition.y;
			float zPos = ((1.0f - touchPos.y) * onePercentTrackPadSize) + (initTrackPos.z - (onePercentTrackPadSize / 2.0f));
			thumbTouchObj.transform.localPosition = new Vector3 (xPos, yPos, zPos);

			// Test For Swipe
			float dist = Vector3.Distance(thumbTouchObj.transform.localPosition, previousThumbPos);

			if (dist > 0.1 && !isAnimating) {
				isAnimating = true;
				float rotAmount;
				if (previousThumbPos.x &gt; xPos) {
					rotAmount = -90.0f;
				} else {
					rotAmount = 90.0f;
				Vector3 currentRotation = uiContainer.transform.localRotation.eulerAngles;
				uiContainer.transform.DOLocalRotate (new Vector3 (0, 0, currentRotation.z + rotAmount), 0.35f).OnComplete (RotAnimationComplete);

			previousThumbPos = thumbTouchObj.transform.localPosition;
		else {
			thumbTouchObj.SetActive (false);
			previousThumbPos = new Vector3();

	private void RotAnimationComplete () {
		isAnimating = false;

To polish this up a bit you could apply an elbow model to the controller, as described in my previous tutorial on breaking things in VR here. An elbow model would give it a more natural movement when the user tilts their wrist. It would also be fun to present the UI on a flick of the wrist, or even press of the Controller’s App Button.

Disclaimer: I’m a Google employee and write blog posts like this with the sole purpose of encouraging and inspiring developers to start exploring Google’s amazing Daydream VR platform. Opinions expressed in this post are my own and do not reflect those of my employer. I would never share any secret or proprietary information.


  1. These are great man…I’m having tons of fun with the demos! Quick question. What would you suggest to keep the controller oriented in orbit around the HMD head position…?

    Thanks again for sharing


    1. Glad you like them. It’s a good question, there are several options: one would be to place the object inside a gameObject that is attached to the y rotation of the HMD and then offset it’s position on the x and z axis slightly so it rotates when the user rotates their head. You could also create this same effect with code adding a slight delay/ease to the movement would give it a more natural feel and mean that it does feel as locked to head rotation as body rotation.


  2. Hi! Thanks for putting these tutorials up. You’ve made a big difference in my Daydream development already. I’m having an issue I hope you might be able to shed some light on. I get this same error for both this and the last tutorial (throwing/elbow model).

    CommandInvokationFailure: Unable to convert classes into dex format.
    C:/Program Files/Android/Android Studio/jre\bin\java.exe -Xmx2048M

    My googling tells me I have a duplicate library file possibly? Or maybe we have conflicting versions of java? Any thoughts here?


    1. @DaydreamerDamo not too sure what the problem could be. Which version of Unity are you on? Do you have the latest version of java and Android SDK installed, and is Unity pointing to this in your preferences? Are you able to get anything running on a phone, like just an empty project? If so you can possibly copy the files from my tuts over to an empty project and see if it runs, not the best solution but a temp work around. Most of the tuts up until this point were built with earlier versions of Unity and prerelease GVR – Daydream SDK, mainly because they were stable. So I’ll get myself up to date too, however the code and the core principles should be the same.


    2. @DaydreamDamo I just posted a new tut on setting up Daydream from scratch If you’re still having problems I’d recommend following the setup and seeing if a simple scene runs, then porting across the files to an empty scene instead of importing the package. My previous tuts were built with the older version of the SDK, from now on I’m going to start using Unity’s native Daydream support.


    3. There’s the problem right there, your Googling. Try a better search engine like Yahoo or Ask Jeeves.

      (I joke, please don’t actually use those search engines 🙂


Leave a Reply

Your email address will not be published. Required fields are marked *