Hello Hypernex fans! It’s been a while since there has been a public facing update, but I assure you, we are still working on improving the current state of numerous projects. That is what we will be talking about in this post. We will go over what we have completed, what we are working on, and what we still need work in (including some assistance) moving forward.
Before continuing, any plans discussing future work is all a work in progress. No promises are made and plans may change in the future. Also, the changes in this document are not implemented in the main build, nor any beta build, they are just WIPs.
IK Improvements
Admittedly, the main focus for the Unity client has been improving the current state of our IK. It is no secret that our IK provider is FinalIK, considering how widely used it is in numerous other Unity applications. While we have had internal discussions about replacing it, it is still important that the current solution works optimally until a new system is implemented. We have made a lot of improvements towards the currently implemented IK.
Fix Head Rotations
Head rotations on all avatars should work correctly on both Desktop and in VR. Thanks @Virtual !
Fix Hand Transforms (Sorta)
There has been a lot of work put into fixing hand transforms, and for the most part, we have them working correctly. Currently, hand rotations work on most avatars, only showing issues on avatars with odd rigs (such as avatars from other engines). Hand positions are also working mostly correctly. It feels that there is a slight offset on some avatars, but it is mostly unnoticeable. This is likely because hand positions are static and not calculated based on an avatar’s transform. For the time being, hands are working much better than before.
An image of @Mochi with working hands!
Avatar Scaling
No, not avatar scaling (we already have this), avatar scaling. There’s a difference y’know! But seriously, avatars used to be forced to a transform scale of (1,1,1). We have eliminated that restriction and fixed the bugs that came with it in our IK fixes.
Better IK Configurations
We have made changes to FinalIK’s IK configurations for avatars and made it look much better. Previously, it would look like your avatar’s hips had its own asteroid belt, but now it respects basic anatomy and does not look ludicrous.
Network Fixes
We fixed an issue where tracker roles would not be respected over the network.
now for our ever so dreaded section…
Current IK Issues
While we made plenty of improvements to IK, there is still so, so much wrong with it.
Some Rigs are Just… Broken
We have no idea why. Some rigs just do not work with FinalIK when they have no reason not to.
An image of me facing forward, slightly bent down, but that’s not what the avatar is doing…
Feet/Pelvis do not Respect Transform Changes
Simply put, sometimes the feet and pelvis will not make be rotated correctly, similar to our hands issue. We believe that the same rotation offsets need to be applied to the IK Targets on the target trackers.
CalibrationData is not Correct with Local Calibration
When completing a calibration (primarily with FBT) the calibration locally does not sync over the network correctly for some avatars. This seems to be an underlying issue with FinalIK’s CalibrationData, but it could also be a poor implementation on our end.
Broken Avatar
Trackers are Offset over the Network
Positioning for trackers are offset over the network slightly. This could be an issue with avatar scaling, message compression, FinalIK calibration, or anything else (who really knows).
Local Avatar Randomly ‘Flashes’
Sometimes, the LocalAvatar will randomly flash in and out. It is so fast that it is barely noticeable locally and definitely not noticeable globally. It is unknown why this happens.
An example of an Avatar Tweaking
A Final Note on our ‘FinalIK Manifesto’
We are a small, ₛₘₐₗₗ, team and we have lots of issues with timing due to us going outside and touching grass (so sad…) and we graciously ask for any and all possible contributions towards getting FinalIK working! If you know anything about FinalIK and have questions about integrating, please do not be afraid to reach out to us! We have a discord and a support email.
finally moving on,
Video Player Improvements
We have also been working hard on improving our video players in Hypernex. We have implemented some substantial improvements regarding handling video formats and dynamic loading of video players.
VLC Audio Source
We now have a working PR of VLC audio being output directly to a Unity Audio Source. This will support having 3D audio sources (with SteamAudio support) on any videos loaded with VLC. There are still some slight issues (such as slight audio desync) and untested features (like multi-channels and audio rate). Here’s a demo of our VLC audio source!
VLC also has some issues with initial loading, on some clients it will instantly crash their game when first loading VLC (sorry @_spidget !)
We also fixed an issue where some files would not be registered correctly, but there’s still some issues with some livestream types (specifically .mpds).
FFmpeg Support
@Virtual has been working on a fork that adds ffmpeg support to the Unity client. It works fairly well but has issues with audio desync. Please help contribute to this project so we can help make this more mainstream!
Other Fixes and Improvements
Below is a list of some smaller fixes and improvements we have already pushed to main.
- Fixed Microphone at Smaller Framerates
- Spawnpoints now fallback to the World Component’s GameObject if left Empty
- All TextMeshPro Texts have support for Emojis
- Fixed NetPlayer Scaling issue for Avatars
- (Sorta) Fixed Outline Shader
- Still some issues with certain meshes
Future Work
Moving down the timeline, there are some huge changes we have been working on that we wanted to share a little early. Obviously, Hypernex strives to be as optimal as possible, but a lot of our code is unoptimized and our stack is pretty poor. We are working on fixing this with some new features.
CDN
We are currently working on a CDN server that will allow hosters to distribute content (with caching) across multiple servers, reducing the load on the real-time API server. This is still in early stages and is not complete, but we do want to complete this as serving content from one server is not a good design.
Hypernex.CCK Rewrite
We are also working on redoing our current CCK to fix issues (such as with uploading) and improve simplicity. As for now, we are not going to share much more on the new CCK, but please note that it will have major changes in how everything is done and will be shipped with a huge update that is still in the works… 
Closing Notes
As always, thank you so much for everyone who sticks around with this silly project of ours. I love seeing and hearing people talking about it all the time, and I love working on it in my spare time. Please know that we are all busy people with other things to do and we try to get as much done as we comfortably can. Remember that anyone is welcome to contribute, even if you don’t know how to code! There are plenty of ways to contribute. 
Thank you all for the continued support!
from Hypernex
