For the past week I ended up revisiting a problem we had earlier during the semester. This problem involved the leap motion hands being visually offset from the player's real hands. This posed a huge problem in terms of UX where players where having a difficult time grabbing the booyos because of the visual disparity in terms of distance between both sets of hands. Most of the users end up taking quite some time before they can get used to it. Evidently this is one of our biggest problems in terms of leap motion controls.
Earlier during the semester, I managed to somewhat remedy this problem thanks to a blog written by Stereolabs.
Essentially, the way stereolabs is handling this offset is by creating an empty gameobject and using that as the Device origin for the virtual hands. And then during runtime, you would manually calibrate the hands, and then copying the transform component of the empty gameobject, and pasting it back afterwards.
After applying the necessary fixes, it definitely contributed to syncing the hands more accurately. But after a couple more testing, we found out that eventually the hands get offset again and we would have to manually calibrate it all over again. Since this process ended up becoming a bit tedious to keep having to repeat, we decided to only calibrate it before making builds. It was a doable process and we decided to leave it at that. This was the case until we ran into another problem.
The hands get drastically offset depending if we're running the build from the towers or the backpack computers.
After some testing, I found out that the hands are offsetting in consistent values. When running the build from the backpack computer, the hands are always offset by about 5 units upwards on the y axis. Why is this happening? I have no idea. But I just fixed this by adding an option that lets you toggle between the tower and the backpack transform.
Remember our problem earlier where eventually the hands get offset even just a bit? I ended up going back to this for the reason that it still happens despite perfectly calibrating it right before making a build.
I found out that one of the reasons why this is happening is because of the position of the physical sensor on the headset. The Leap Motion's position relative to the ZED mini camera affects this offset heavily.
This means that the relative position has to be exactly the same between from when we make the build and when someone is actually playing the game. This is near impossible particularly for our case because of the fact that we never actually 3D printed a mount, and instead we relied on rubber bands. This means that the both the ZED mini and the leap motion will inevitable be moved one way or another: when the user is wearing it, the rubber bands get loose over time, they get bumped by your hands, simply repositioning it. There's a million other reasons why they would get moved and at the end of the day, it's just impossible to really make sure the positions are extremely accurate from when we made the build. A simple solution could be is to simply 3D print the mounts. However this is only of the reasons, I also encountered other tech issues that causes the hands to get offset no matter what.
In the end I concluded that this is just one of the problems that I really cannot fix. It came to the point that I've wasted one full week of just troubleshooting and trying to find a solution. I pretty much gave up on trying to perfectly sync the hands and ended up brainstorming of ways to design around this. UNTIL... I SAW THIS TWEET!
I messaged Yujin asking how he was able to get such an accurate sync between both pairs of hands. It turns out that we were actually doing something very similar, and that was to use an empty as the device origin, which we would then later position on top of the hands. The biggest difference was that Yujin was manually calibrating it on runtime... using a keyboard. It hit me like crazy, why in the world are we trying to get the perfect sync when we could just calibrate it during runtime. Our setup also included a wireless keyboard that we can just use, so this made perfect sense.
After a simple implementation, there we go, perfectly synced hands!
If there's anything I learned from this experience is that you really don't have to fix it, sometimes you just have to go around it. I mean, this was quiet evident for me even back then, but it never really hit me this hard until I started working on tech that frankly, I have absolutely zero control of.
Comments