[Development Updates]

A very provisional tinker with the reel rendering stuff - getting a bit closer to that thing where you could peer through the glass and see the 'back' of the reel to work out how may nudges you needed!

[Development Updates]

6 hours ago, A:E said:

Excellent, this is looking superb John.  Wow, this is immense, is there any latency issues at all?

J

Thanks man   Good question re latency...

So I think when I last checked I had latency down to the minimum it could be with this particular technique of window capture (1/60th of a second, i.e. a single frame behind what MFME is displaying).  This is inherent in this specific technique of data capture, as MFME must draw, then be captured and processed, and then redrawn by the renderer, so the renderer must always be a single frame behind at best.

I did have another idea though, so I might be able to get around this... but not a priority for the moment.

The reels are actually slightly more behind when using the physics 'bounce' as seen in the earlier 2p Cash Nudger tests, as I did a simple model for those that is always 'catching up' to MFME  (though it does look very similar to what reels do on actual 'bouncy' machines in youtube videos, so I think it's close to what it should be).  Reducing/eliminating bounce effect would reduce/eliminate the reel lag.  More ideally, me writing a tweaked reel physics model (that only applies physics when it detects the reels have abruptly stopped), would probably be able to eliminate some/all of that additional latency though and retain the nice bounce, so not worried about it for now  

TL:DR - there is technically lag, but pretty much imperceptible at 1/60th of a second (a single frame behind MFME on a 60Hz display).

[Development Updates]

Got the other 'On' lamps in - so now Monopoly Millionaire has the red lamps working.  Also got in the rom-controlled variable brightness lamp support, so in the video, you can see the lamps fade up/down when entering the inner board feature, and at game over points from the board  

Looking into reels and getting the reel windows transparent next, to get these initial converted flat layouts basically playable...

 

[Development Updates]

A little progress on the new MFME automation converter - extracted the basic On1 lamps from the Millionaires Row layout via the new tool and hooked up to a minimal data layout  
(I noticed there were no game sounds when recording - nothing to do with the converter, prob just something random on my PC!)
 

 

[Development Updates]

7 hours ago, slasher said:

That's cool. Still curious to see where it chucks them all together haha

lol me too - the first working Monopoly video was a 'cheat' of sorts, by using specifically an 'unpublished' layout and using the MFME lamp export feature available on unpublished layouts. 

This new approach when complete will be good for all layouts, plus will get much more complete info about the reels, segment displays etc.  Maybe have working lamps in their original (2D) configurations for all layouts by next week, if I keep getting time to work on it  

[Development Updates]

Have managed to get success with new approach for extracting numerical data  

This works by clicking in each number field, then sending Ctrl+A (select all), Ctrl+C (copy), then parsing the clipboard contents.

Not a super exciting video, but I thought I'd upload for those interested in the technical side of the 'converter' aspect of the project - here it extracts and prints:

z order, component type, x, y, width, height

...for all the components in Millionaires Row, using the new reliable technique  
 

 

[Development Updates]

Made some progress on 'reading' the drawn text from the MFME window to extract the layout data, though the AI isn't up to the task of interpreting it!  It makes mistakes. 

So backup plan is to roll my own basic screen text scraper for the specific MFME number font, as it looks like the font is hardbaked into MFME, so it shouldn't be too bad and won't break with new versions of Windows etc.  Bit of a ballache, but needs doing to get guaranteed 100% accuracy, otherwise there'll be random lamps in the wrong places!  

Mistakes marked in pink, also it sees "21" as "ral" every time:


Edit: ok I've just had another look at this - new plan   I can use my automated mouse clicker code to double-click the number fields to select the entire contents, then send Ctrl+C to copy them, then read them from the clipboard - phew, that sounds like less work lol  

[Development Updates]

2 hours ago, mattrickl06 said:

Wow just read through all this - its insane. Keep up the great work. Maybe we could play one via Oculus at some point?

That is the hope (along with normal PC monitor play)  

[Development Updates]

Great to hear it's considered a good game - I chose it as the new test machine as it's got a pretty complex layout; lots of VFDs, dual red normal/reel lamps and a DMD... oh and 'On' lamps on background.  Plenty of stuff to solve  

[Development Updates]

No eye candy here, but a tech update   I've started working on the 'MFME Automation' stuff, to allow me to extract complete layouts with every image/setting/coordinate etc.  The technique is mixing the window capture stuff I'm using anyway to run the machines, plus simulating mouse/keyboard input.

This vid shows it performing the steps:
- launching the test layout to convert
- skipping the initial popups
- pausing the machine emulation
- enable Design Mode
- move mouse to top-left of layout
- right click on background and select properties
- capture properties window
- move mouse to 'next property' button in bottom-right of window
- click next property button every quarter of a second

Bit of a bonkers approach but it seems to be working well, so I'm gonna go with it  

Next up will be some cleaning up, then bring in some (more!) AI to interpret text and numbers drawn in the window.  Once I've got that working, I can start on extracting the full data (bitmaps, x, y, width, height, config values etc) per component that make up the layout.

 

[Development Updates]

6 hours ago, A:E said:

Wow, JP.  We could create Smithy's all over again bud      Literally everything will back then in the early 90's would be able to be emulated.   Hatfield at the top of the stairs looking down on his domain ;).   Getting you to play machines for him, then nipping under the clock for a beer

J

Would be nice to have a simple editor at some point so people can configure their own arcades - I'd certainly make a couple of the Cleethorpes ones as I (very vaguely) remember them from my youth    It's a shame camera phones weren't ubiquitous back then like they are now.  I guess there might be the odd photo or two of some UK arcades from the 80s/90s buried on the internet somewhere....

[Development Updates]

On the virtual arcade situation - without wanting to over-promise anything, as this is just a hobby project I'm working on - that's kind of where I'd like to see this ultimately end up, having seen how well the Cash Nudger worked in the 'holodeck' environment. 

I think the big part for now is working on this 'semi'-automated layout conversion/upscaling, since an arcade with 50 different fruit machines is more fun than just 50x "2p Cash Nudgers"!

Other machine types (as we saw from the Pacman test) are possible... along with physics-based crane games, coin pushers etc (as I just use Unity's PhysX engine).  The trick I'm thinking is to have an 'interacting/viewing' mode... so you can only interact/view a single machine (in full 8K with emulation/physics) while you're walking around.  That way, it keeps the emulation/simulation overhead down - and having made a prototype coin pusher simulation a few years back, doing all the physics for the coin colliders adds up fast.

So the other machines would play attract mode loops rendering with normal (non-8K) textures, but if you have one centred in your view and are close enough to it, then it would switch to showing what the machine is actually doing (if it's being played etc), with the 8K textures.  Keeps the network bandwidth down too, otherwise if the arcade is busy everyone is transmitting shit to everyone, even for machines the other person can't actually see.

Something like that anyway - but it's all a long way off - just got to keep breaking off bite-sized pieces and solving them one day at a time  

[Development Updates]

6 hours ago, Big J said:

Fantastic JP but what I want to know is do you get any sleep? This looks one hell of a task that you make look quite easy (obviously far from). Loving the updates and so looking forward to the end result

I'll be honest I only get around 6 hours a night tops these days, what with my day job and working on this new project! 

The 'end result' is a very long way off though to be honest, road map is currently something like (in only an approximate order):

- R&D stencil shader or some similar approach to allow 1x compressed masked texture per MFME light layer + 1 bit common mask texture, excluding common redundant Off layer

- semi-automate 2d layout -> 3d perspective-corrected sub-panels

- automate AI upscale layouts once shader implemented

- R&D new LOD techniques to allow 8k when close, standard rendering at distance so graphics card requirements stay modest

- recording lamp/reel/other output data to capture efficient attract mode loop per machine, optimise for only frame deltas - to allow machines to run in attract without MFME instance

- R&D automate MFME complete layout extraction to open format, via send clicks, send keys, and OCR for screen-scraping.

- automate reel creation, gradient removal, alpha window cutouts, cylindrical band mesh with interiors, symbol offset, scale+position

- automate creation of associated (more compact, accurate) data layout 

- R&D alpha vfd/dmd capture/modelling/rendering approaches (use quick n dirty screenscrape initially to keep things moving)

- potential automate direct capture of data layout via memory signature (to bypass ASLR) instead of window scraping, to save GPU bandwidth and perhaps reduce latency

- creation/cleanup of 3d cab models (may be able to get a 3d artist to help on this)

- Networking R&D: server vs P2P?  potential mixed solution, perhaps use twitch API for video screen element broadcasting?

Good grief, that is awful lot of work now I've written it out!    And there's always like 2.5-3x more stuff to do than you anticipate on something like this...  Ah well, I'm just gonna keep chipping away at little corners of it, see where I end up

[Development Updates]

There's a lot of hacky stuff here, so it looks further along than it is - but all the lamps were automatically positioned/scaled/textured/indexed.  This was done from an 'unpublished' layout that enables a 'Lamp Export' option.  This doesn't give much, but gives enough to make a start for basic converter development without getting too sidetracked.  

For the future I plan to 'automate' MFME so that it loads up the layout, goes in edit mode and saves out each resource along with its data - allowing for more conversion options (reels, vfds, button mappings, transparent reel windows etc).  But that will take a while to put together, so I may carry on testing with some 'unpublished' layouts for the moment with the manual 'Lamp Export' option, plus I need to write some kind of shader that I've no idea how to do as this is rendered very inefficiently, so a helluva lot to figure out yet...  

 

[Development Updates]

Not so much eye candy in this video update, but a good milestone for me at least, after a few nights of battling with Unity!  I have a glimmer of at least some beginnings of automated 2D -> 3D layout conversion...  

 

[Development Updates]

36 minutes ago, DrB said:

This is really cool. As a suggested future use, could it be used for a 3d tracked virtual cabinet like the vpin stuff ?

In case you have no clue what I am on about, there's a guy demoing his vpin with head tracking off/on, which would be great on a bottom screen of a 2 screen vfruit cab. You can get a ps3 camera for £4 from cex and the rest looks like lego and some cheap leds.

 

I am indeed up on the 3d tracking vpinball stuff, I did a couple of bits on the standard vp code many moons ago   

You could render the top/bottom glass panels & reels into say two monitors - one for top glass, one for bottom glass - to use this renderer in a standard cab-optimised scenario, and get good looking results (due to the upscaled texture stuff, reel/lamp physics etc).

However the problem would be the head-tracking part.  If you wanted to get in close for a reel peek, Kinect/eyetoy etc wouldn't be able to track your position accurately, as it needs a good unobstructed view, with some distance between it and the front of your face.  Some people have also rigged up things like sunglasses frames with LEDs and Wii sensor bars, but again the tracking wouldn't work with your face so close to the glass.  If you tried using a Wiimote very close to a TV you'd run into similar issues, due to the sensor bar receivers not being able to track the Wiimote led emittors.

This'll get good results in VR with a headset - however primarily I'm just making it to be playable on a standard PC as that's what most people have (the VR stuff you practically get for free programming-wise, once you've made a 3D environment).  So people can walk around the arcade and play/watch machines, and if they have the VR headset then it's almost like being there, apart from the smell of stale cigarettes!

Also - the only real benefit from the eye-tracking would be the subtle shifting of the 3D reels, and the slightly raised buttons.  As most of the machine front is usually flat glass panels (with some notable exceptions like the excellent Pyramid), it wouldn't change even when tracking your head position correctly.

[Development Updates]

Another update - I've generalised the emulator handling a bit, allowing me to add a hacky version of MAME support.  There's a slight input lag, that will be fixed with compiling a special version of MAME that outputs it's graphics directly in a way I can display on the same frame without the slight delay.  Here's a demo, I didn't do very well, but I'm pretty tired, that's my excuse anyway   

Further improvements would be a CRT shader and of course I would clean up the cabinets so they are clean/sharp/properly lit.  Fun early proof-of-concept though   

 

[Development Updates]

4 hours ago, A:E said:

Yeah, echoing others comments mate.  This is looking superb, can’t wait to try it out... if you get to that stage, no rush but this is pretty out there.

Can materials have reflective surfaces like real glass etc JP?   This is so cool dude

J

Still too hacked together for try out builds just yet I'm afraid, though it will get more stable.  Currently it's pretty aggressive, forcing itself on top of all the other windows among other things, and there's more hacks to go in yet to make it all hang together properly (for machine switching etc).  I have a plan though - once I've got all the hacks stable enough, I'll work on some lifecycle management which should hide all that hacking away  

And yes - you can do reflections/lighting - though the reflection would probably be more of a 'cheat', like a prerendered cubemap rather than realtime - I'll experiment in due course  Also on the Family Guy above, I'm using some specular shading to 'catch the light' on the yellow trim.

[Development Updates]

Made some progress - MFME now hidden while playing  Also - there's a WIP cabinet model I'm playing with in the background.  Bought it, and I think I'll be able to adapt it in Max to create a variety of different cabinets, to put the working MFME 3D machines into (Family Guy is just the static demo textures it came with).

So the flow is now:
- launch Renderer exe
- it sets itself to always draw on top of everything, and hides the mouse pointer (System-wide)
- it loads MFME and the data layout
- it listens for system-wide Key input (as it no longer has focus, since MFME needs it for the key inputs to play the machine), if it gets an Escape, it kills MFME process (ungracefully) and quits

It's all pretty hacky, nature of the beast really - hopefully with another day on it I can resolve the various issues, and minimise the hackiness.

It is nice to see it launch full screen though with no MFME window on top obscuring stuff  

 

[Development Updates]

8 hours ago, Reg said:

This is really good - perhaps one future is keeping these files within the layout directory as well for the machines that could be shared when people create them.

The future is certinally bright here.  If at some point in the future you want your own forum area for this if you choose to go all the way so to speak, consider it done.

 

I'm not sure what the file format would be yet, for the 3D layouts - probably text format Unity scriptable objects for various data, along with textures/materials.  But they could easily be packaged along with original source MFME layout (or visa versa) - I've honestly not thought that far ahead.  Still in the exploration stage of how what can be done, but would be totally open to what works best for the scene 

Thanks for the offer of forum area - I'll just keep popping updates in here for now while it's just a tech demo I'm tinkering with, but if it gets past some of the other hurdles that need figuring out and starts to get to a usable platform, that'd be great  

[Development Updates]

53 minutes ago, Reg said:

So cool - I wish Chris has been around to see this.

Stupid question - but hey, let's go with it.

What is the end goal for this ?

Will it ultimately render machines based on what layout is loaded within MFME with auto-detection doing the backend work for the glasses and the lamps.

I suppose what I am asking - is MFME the engine behind the magic you are doing with the rendering and in the future - will it be plug and play for whatever machine you load ( perhaps after some initial tweaking of settings ) ?

Sorry for the dumb questions.

 

I know - it's a shame the timing   I only got back into this stuff as I signed up here to pay my respects in the Wizard's Rest thread, then started thinking abut FME again.

End goal wise - I don't think it could be fully automated, it would be more that there would be a 'layout format' like MFME has, so layout artists could make new ones from scratch, or convert 2D MFME ones (like I have done here).  I still have some fundamental stuff to research/code, in order to have the 3D view full screen, and MFME hidden behind but still receiving input, though I have a rough strategy for that which should hopefully work.

The rendering is indeed driven by MFME (it could also be driven by MAME, though that has far lower compatibility compared to Chris's amazing emulator).

I'm still playing to be honest   ...here's another take on the camera - with the machine in an environment that can be navigated
 

 

[Development Updates]

Done the top glass, so a little closer to a full machine  

Made a very minimal start on the 'auto-director' idea for the camera.  Final camera pans will be much nicer (damped at start and end, position as well as rotation movement), this is just bare minimum to test. 

So the director looks for specific lit/off lamp combinations to know when to switch to different camera angles (there are only two angles in this early test - full bottom glass and full top glass).

It was pretty stingy with the nudges, but got a couple of sets in the first few credits on this vid so you can see the basic idea  

 

 

[Development Updates]

9 minutes ago, A:E said:

Is it possible to use the actual reel symbol as a lamp mask with maybe some glow bleed?  This is for future thoughts mate, I know you are working on the primary workings before you polish bits

Totally love this though mate, really interesting stuff, fantastic skills

J

Yep mate, with a shader it's possible to do specific stuff like that    For instance you could have a shader to turn all green-ish pixels in an image black, or change all pseudo-silver coloured pixels to have a reflection of the room/lights behind etc.  So configurable custom shaders will be the ultimate way to go for some of these effects.

To the layout designer though, when the shader is done, the coding will be hidden away   The parameters will just be tweakable in realtime with named sliders to configure the desired look for that machine (e.g. Glow Bleed 0 --------|--- 1 ). 

[Development Updates]

I've had another bash at the reel lamp effect.  I went a bit overboard on the 'warmness' of the lights (these Tuppenny Nudger type machine's reel lamps seem to be a lot brighter/whiter on the youtube videos)... but I do like a nice cosy bulb glow!  

In the long term it'll want a custom shader, that will give a much better effect than this quick job, but I'm just getting things 'good enough' for now, so I can keep progressing the prototype.  

 

[Development Updates]

1 hour ago, kinkyjohnfowler said:

Really interesting to see this.

Not sure if it's already been explained and I've just missed it/it's went over my head, but in layman terms what exactly is going on here to make this work? 

EDIT: I assume MFME is being used to emulate the machine and a secondary tool is being used to display the layout, but just have no concept of how this being made to work ha

Hi  

So how this currently works is by scraping the contents of the emulator window.  You'll see in the bottom-right there's a 'data layout' - I am capturing the MFME window content each update, deriving the emulated reel step positions and lamp matrix state, then hooking up my 3D components (reel, lamp etc) to be driven by the scraped inputs.  How the data gets from MFME to the engine may completely change but that's a conversation for the future.

I could also make a fruit machine 'data layout' in MAME (which emulates quite a few machines from some work done a few years back), and it would work in the same way - so it's technically emulator-agnostic.  The primary focus is on MFME though  
 

53 minutes ago, A:E said:

 

So are you able to change 3D orientation while you are playing, or is it fixed perspective?   Really immense work in just a short time!!

J

Yes, that's planned (smooth panning between topglass view, bottom glass view, reel peek etc)    This will initially be just skipping through views with say pressing Tab key... but later may also have an 'automated Director' option, constructed per layout based on rules 

Ie: 

If Stop Nudges Top Glass message lamp flashed, and no other lamps flashed in sync:  Trigger pan to top glass  (as the system has detected time to stop nudges)

...then...

If Nudge buttons flashing and on top glass, trigger pan to bottom glass (as the system has detected time to use nudges)

Something like that could probably made to work pretty well, to save the user having to manually tab.  With the option to Tab at any time to override what the auto-director chose.