Development Updates

[woodsy]

57 minutes ago, johnparker007 said:

Tech update: A follow on from video above, dumped a load of these cabs for a quick stress test - looks like it's going to need significant work to reduce CPU/GPU load on simply playing video streams (this is without my planned arcade audio system on top!).

First thought is to have variable frame and/or resolution video set of the same attract.  So a machine far off in the distance could be playing it's attract at 30 or even 15 FPS (perhaps with some shader magic to blur between the frames to hide stutter). 

And also to have low resolution videos, so if a game is at 320x240, there could be attracts at say 320x240, 240x180, 160x120, 80x60, to be shown based on how far away from machine you are.  Though the challenge would then be to not get stung by overhead trying to resync videos on playback as a machine becomes closer and needs to switch res/framerate (as you walk towards it).

Complete unknown if this stuff is actually possible!  But I did really want it to be able to have lots of video games on screen at once without such heavy CPU/GPU usage...

Just trying to add ideas to the mixing pot, what about a collection of screenshots unique to each machine as a photo gallery type thing sideshow for games that are not within a certain range from user that depicted when the player approached, is that better for cpu/gpu use, I don’t know  l anything but just an idea as I believe video takes more than a gif? JPEG slide show? Sorry if it’s nonsense but I’m just trying to help  

[johnparker007]

4 minutes ago, woodsy said:

Just trying to add ideas to the mixing pot, what about a collection of screenshots unique to each machine as a photo gallery type thing sideshow for games that are not within a certain range from user that depicted when the player approached, is that better for cpu/gpu use, I don’t know  l anything but just an idea as I believe video takes more than a gif? JPEG slide show? Sorry if it’s nonsense but I’m just trying to help  

That is actually my backup plan, for both low end PCs As I think they will struggle with the videos even if I can optimise them.  I was thinking I'd end up with some kind of 'video' settings, so the user can choose between less videos/more performance up to more videos/less performance.

(so to answer your question, yes video costs more than still images).

This is early days in the experiments though   The optimisations for fruit machine rendering took months, but it ended up many times faster (I originally worried it would not be possible!)... hopefully the videos will take much less time to get to a usable speed, it's not as crazy as all the custom GPU rendering stuff for the fruities  

[johnparker007]

One part of the puzzle is solved in that, since updating to the new Unity version, the Editor performance is just a lot less than the real build performance.  Currently it's not a very fast process for getting test builds and they can't be profiled... I may have to look into getting it running as a debug build for future optimisations.

CPU/GPU just about in usable range, these videos are high quality, I can probably optimise more... there will be more to check later, in terms of a larger arcade 200-300 video games (I'll just use repeats)... and culling/resuming from different points to try resync to attract audio... audio is not for a while, but the sync will be an issue to figure out...

 

[Reg]

That looks so cool as the attract screens fade in the distance.

[serene02]

Yeah that looks sweet as!!

J

[cja272]

Looks great

[johnparker007]

It's definitely showing promise   Very long-winded to profile since the Unity update (required for low lag MAME in the Arcade Sim system) - but I've a couple of ideas to get that GPU load down, I think (hope) it might be basic model issues, as I've still quite a bit of work to do on the Goliath yet, more performance tests to follow...

[Danik2343]

Add a save state function, I don't know if MFME supports this function, but MAME supports it, it's better to make this option experimental.Результаты перевода

Will there be arcade game machines added? It would be interesting.

[johnparker007]

2 hours ago, Danik2343 said:

Add a save state function, I don't know if MFME supports this function, but MAME supports it, it's better to make this option experimental.Результаты перевода

Will there be arcade game machines added? It would be interesting.

Save is planned, along with potentially a couple of other nifty features ...though for now I am just seeing if I can get it all to actually work in a scalable (say 300x video game machines) way... these are just experiments for the moment...

[johnparker007]

Been spending a fair bit of time trying to work out a good workflow to get the attract vids from emulator through to trimmed and Unity-ready.  Annoyingly I have a feeling I used some random online converter, my old attracts from many months back are working much better at a small filesize, than a reliable process I'm setting up now!

I well I'm sure I'll be able to sort it out with more fiddling...

Anyway, also remembered that high GPU usage, and good news is a chunk of it is the Goliath model as I suspected, and not the GPU video decoding

Removing models takes GPU% from 43-44% down to around 28-29%.  This will be fixable with more work on the model, so the scene here should end up running around 33% GPU on my NVidia 1060, which is fine.  For users on limited hardware there'll be some kind of setting to disable videos/play less videos.

Reduced GPU usage:

 

[johnparker007]

Tech update on the above encoding issue - cracked it   And made a careful screenshot notes of all settings.

So the workflow will be a quick tool that I knock up, that given a list of mame rom names ("salamand" for Salamander etc), will:

- run MAMEUI from commandline, passing the relevant args to create a lossless video (with lossless audio).  1 minute is around 750mb, so pretty beefy files.

- then it will control Handbrake from commandline to create a very high quality compact video file encoded with these good settings

- then I'll use avidemux to create a short 'perfect' loop via keyframe/black frame/scene change alignment depending on game... that loops 'properly' but will probably only be around 2-3 mins depending on game so say 20MB per game with decent quality 60FPS video/audio)

I've made a test very short video of Vulcan Venture, thru the above pipeline (perfect loo point was quickly find via keyframes) - so this shows a 'perfect' loop, of one of the levels shown in attract - video length is 1:10, doesn't show repeatedly looping cos I'm knackered!  Loop point is around 0:48 just before 6,7,8,9,10 high scores disappear - if you look closely you see the stars change.

So many frontend setups have those awful compressed videos at daft 24 FPS - hopefully we can put that behind us   Like the fruit machine attracts, I'm hoping to make the video game attracts indistinguishable from live emulation.

Edited January 27, 2022 by johnparker007

[serene02]

Impressive work mate

J

[johnparker007]

I realized I could now crank my good settings... only used one game Vulcan Venture as a test so far - but this is the source file that is 6 minutes long, and 20.3MB. So this translates to:

3.83MB per minute for 60 FPS near-lossless video quality, with 160 kbps AAC audio.

3.83MB per minute!  More tests required, but that is crazy small for the quality  

Here's the sample 6 minute file, for the technically curious - recommend playing in VLC over Windows Media Player, it seems to look more accurate in VLC (same as it looks in Arcade Sim):

 3.83 mb per minute AV 60FPS.zip


(Just mathed it, works out at around 65 kb per second, for the 60FPS near lossless video - hopefully this will work ok on my ATI Radeon test PC).

[Retrospacecadet]

Wow, not been on Desertislandfruits for a while and have to say reading through the 95 pages of this thread has left me in awe.  Great concept and even greater skills to make it happen.  Fantastic Work 

Edited January 27, 2022 by Lampit

[johnparker007]

Tech update  

Managed to get compressed recording sizes down by about 25% whilst slightly improving quality (I didn't have the encoder set to run at its slowest speed, slowest speed gives best results).  Now at 16.5MB for 6 minutes, so 2.75MB per minute (for Vulcan Venture, other games will be larger/smaller depending on how much the screen contents change).

Sample here for those curious to see the compact goodness I think these are now optimal all-round settings for compressing MAME videos to h264: 
Vulcan-1(22 slow encode).zip

And have spent tonight battling MAME Lua commands to get my 'Video Builder' internal tool started - now can record direct from the Unity project video game definitions, and generate the initial lossless recording stage  

Quick demo of that process record 3x ~30 second test videos and playing back:

I'll be setting a bunch of these to record tomorrow while I'm working, so I can develop other stages.  I suspect I can get them recording unthrottled from MAME, so then a 10 min lossless source video might record in around 30 seconds on my machine... which would be handy.

Edited January 27, 2022 by johnparker007

[johnparker007]

Tech update (a lot of these to come with developing the video game stuff)  

I now have the first part of the video processing pipeline done.  Took longer than expected due to a real nasty bug in Handbrake CLI (not present in GUI version) - managed to make a reliable workaround... so all good.

So for the 'pre trimming' stage, it: 
- records lossless from unthrottled MAME for a specified approximate amount of 'emulated time' (90 seconds in this demo)
- encodes with my high quality compression settings and writes out
- deletes the very large source file
 

(sound when played back in VLC is tinny/distorted due to plugins I have so my VLC doesn't disturb neighbours when watching late at night, the .mp4s actually sound fine  )

Next up is perfect loop creation, offset storage in machine config, generating final .mp4 loop...

Edited January 29, 2022 by johnparker007

[pubjoe]

Exciting stuff!

[johnparker007]

Well I've been working on this solidly since Friday night lol   I've not made short (usually around 60-90 seconds) 'perfect loop' high quality attract videos through the new pipeline I've developed.

As expected, without optimisation experiments, this is too much for my PC to handle, but I've edited this to a run where over half of these 50 or so machines didn't freeze up, just to show some of the progress:

I'm hoping to be able to do something so I have progressively lower resolution video as they get further away, and can switch between them without stalling everything.  Also the sound will not be handled like this, there's a separate system planned if I can figure out the syncing...

I don't know how possible this stuff is, I googled around, I can't see anyone trying to do anything like this in Unity... so much like the fruit machines, it's all highly experimental, no guarantees I can make it all work on current hardware!

The dream is 3d virtual space sounds, and keeping the native framerate videos, and developing a video lod system, so this can all run on a half-decent PC... but I may run into a showstopper issue yet...

 

Edited January 30, 2022 by johnparker007

[pubjoe]

It's awesome that you've applied CRT curvature.

[johnparker007]

23 hours ago, johnparker007 said:

...but I may run into a showstopper issue yet...

Narrator: He ran into a showstopper issue.

Ok so a (fairly technical!) tech update - I think it's impossible to do the idea I had in mind to allow for many more videos onscreen at once... which is unfortunate  

The original idea was to do a v3 of the new video pipeline workflow to make what I guess I'd call 'video LODs'.  So as machines became further away, and so smaller onscreen, I would switch to a lower resolution video.  So there'd be a video at 1/2 resolution of original, then 1/4 resolution of original, then 1/8 resolution of original for distant rendering - that would have much lower CPU/GPU costs, but still look the same.

By switching between these as a video game's screen crossed distance thresholds from the camera, it'd mean even though there could be say 30 game screens visible at once, a lot of them would be off in the distance, so playing back a mini-video file of say 32x28 pixels (1/8 the full resolution of Salamander's native 256x224)

But, to have these really nice quality videos, I am using a h264 codec.  It's efficient, and so the videos only have infrequent 'keyframes'.  A keyframe is a lossless image of a frame, and usually created by the encoder when lots of pixels on-screen change at once, so future frames have a clean reference.

When I inspect Salamander, it has a few non-standard keyframes, but most are occurring at 10 second intervals.  So for a 'random' seek (as we'd see in real world use in Arcade Sim as the user walked around, and videos changed resolution), it is very expensive indeed! 

For example: the user walks forward, so Salamander crosses a threshhold, to become close, and need the next resolution up.  Say this was at 41 seconds through the attract video loop... and the keyframe prior to 41 seconds was at 32 seconds - that means the encoder has to:

(1) Jump to the earlier keyframe at 32 seconds
(2) Play the following frames as fast as possible without displaying them, until it has played through to 41 seconds
(3) Then it can play normally from that point it has now decoded up to

Steps (1) and (3) are fast and efficient.  Step (2) is is a monster, very slow and GPU decoder intensive (this is already performing hardware-accelerated decoding using the GPU).
 

I've checked the 'other' Arcade Simulator - 'New Retro Arcade Neon'.  Their arcade layout seems designed to prohibit there being many games onscreen at once, but I need completely open, and user-controllable designs.

Also, in terms of sound, they are not using a separate arcade live ambient sound approach, but instead the game sounds fade away very quickly as you get just a couple of meters away from a machine (so that they can pause the video, as pausing, then resuming from that same point at a later time is very efficient...  it's just seeking around, not hitting keyframes, that is the issue.

So that's a way that I could also do - but I'm not really feeling it...

I can force more keyframes, but that bloats up the video file size rapidly, and still doesn't really solve the issue, just reduces it a little.

I suspect I'll end up having to bite the bullet and do it the way the New Retro Arcade Neon guys have... but that's a last resort, plus doesn't play well with the open plan arcades I want to support... the problem was that I wanted to have the sound audio for around 24-28 machines at once in a decent radius around the user, and share the ambient audio system between both the fruit machines and the video games... as making a convincing version of that 'arcade sound' was always a dream aspect of the project... and the fruities generally will only be making sound when a bot, or another player in online mode is playing.  But the video games make a lot of classic sounds during attract.

So - that's where we're at anyway... I will have to compromise somewhere I think.

Edited January 31, 2022 by johnparker007

[johnparker007]

Tech update   I'm very relieved, I have found a way around the very worrying  'showstopper' issue I discovered yesterday  

So to summarise - I have an idea to make the video rendering much more efficient, but I need to be able to get around the very high seek times.  Well I'd discounted using additional 'keyframes' as impractical... but apart from a small quality hit, it does allow me to (massively) reduce the seek times, so that they are at a usable level (there is also increase in filesize, mentioned below).

Originally, they were completely stalling the GPU decoder, which mean all the other videos that were trying to play would also stall, and everything went horribly wrong!  

Here's a before/after video of a test I've made to be sure.  The test show a cube move up on the left repeatedly, whenever it gets to the top, it tells the bottom row of videos to start playing from a new random time.

Original - massive seek times breaking everything:

After: with a very high number of force keyframes, I can seek at a usable rate for the 'VLOD' idea:

Very relieved, this is total codec abuse hehe, but it does work for this purpose.  The compromises are:

- a significant increase in video filesize; each new version of an attract video will be around 150% of it's original size (Starforce test video went from ~3MB up to ~4.5MB).  This is addition to the extra 1/2, 1/4, 1/8 size video files that will be generated to make the video LODs for each game.  As bandwidth costs continue to fall in the coming years, I don't anticipate this being a massive issue, considering the result.

- a slight reduction in quality, in terms of noise edge artifacts. The videos can now stay at the all important 60FPS however, and the smooth movement is very distinctive of arcade games (I am considering a slight tweak to bring arcade games that run slightly off to run at 60FPS exact for the attracts, so everything will be smooth, as synced with the monitor, even 120Hz monitor/VR).

So fingers crossed I'll have no more showstoppers on the way to at least trying to make this 'video LOD system' idea... though it still might not work, I can't find anyone doing anything like this online, so I think it's probably never been attempted!  All a bit of a gamble lol  

Edited February 1, 2022 by johnparker007

[bionic_beast]

Well now I've got a new problem.

Arcade Simulator no longer black-screens on boot up and I can load in the arcade just fine, but now when I try to start any machine...

Game slows down drastically.

Edited February 2, 2022 by bionic_beast

[johnparker007]

26 minutes ago, bionic_beast said:

Well now I've got a new problem.

Arcade Simulator no longer black-screens on boot up and I can load in the arcade just fine, but now when I try to start any machine...

Game slows down drastically.

This is down to the unstable (for some) window scraping and/or dll injection I have to use to use MFME for the fruit machine emulation.

When this happens, if you Alt-Tab, and switch to the MFME window in the background, and close/force close that process.

Then in ArcadeSim, press escape to exit the machine.

Then try the machine again - does it happen every time?

I seem to recall there was an issue on your machine, to do with not having permission to write a particular file - speedhack2.dll? 

If the machines are doing this every time, if you could click up in the topleft corner 3 times, it will bring up a debug console - please post a screenshot of that here, it may provide some clues - ta  

[bionic_beast]

30 minutes ago, johnparker007 said:

This is down to the unstable (for some) window scraping and/or dll injection I have to use to use MFME for the fruit machine emulation.

When this happens, if you Alt-Tab, and switch to the MFME window in the background, and close/force close that process.

Then in ArcadeSim, press escape to exit the machine.

Then try the machine again - does it happen every time?

I seem to recall there was an issue on your machine, to do with not having permission to write a particular file - speedhack2.dll? 

If the machines are doing this every time, if you could click up in the topleft corner 3 times, it will bring up a debug console - please post a screenshot of that here, it may provide some clues - ta  

Okay, it seems all I had to do was let a machine boot fully (slowdown and all) and the problem is mostly resolved, there's still some slowdown, but it's negligible. (Thanks for the advice regardless though)

[johnparker007]

1 hour ago, bionic_beast said:

Okay, it seems all I had to do was let a machine boot fully (slowdown and all) and the problem is mostly resolved, there's still some slowdown, but it's negligible. (Thanks for the advice regardless though)

Ok I'm glad it's at least 'sort of' working for you!    The slowdown must be to do with MFME scraping/dll injection... very non-trivial to work on unfortunately.