Post subject: Wild Idea : Mouse movement translated to D-Pad Impulses
RGL
Joined: 7/13/2017
Posts: 57
I had this idea for while and tried to give it a test: Basically a tool that gets an average of the current mouse movements and translates it to keypress impulses send to Bizhawk to use with games that have cursors but only support dpads. (Like Revolution X for example) Lets say you define a maximum of 200 pixels of movement to be the limit. So a movement of 200 will result in a sequence of (1,1,1,1,1,1) while 100 would be (1,0,1,0,1,0) and also slower and faster sequences inbetween. With the goal to make the Dpad cursor move smoother. Of course that would highly depend on the game and how fast it lets you move the cursor and how much acceleration there is. Unfortunatly I am a bloody amateur using Delphi. I tried sending keypresses to the Bizhawk window via Postmessage or Keybd_event but couldn't get Bizhawk to recognize the keys send to it. I am not familiar with lua. Could a lua script achive this somehow?
Player (80)
Joined: 8/5/2007
Posts: 865
Ask and ye shall receive! Download Mouse2Move.lua
Language: lua

-- The sign of a scalar. local function sign(x) return (x>0) and 1 or ((x<0) and -1 or 0) end -- Sums over a vector. local function sum(v) local total=0 for i=1,#v do total=total+v[i] end return total end -- Did we just press this key? local function justpressed(keys, last, button) return keys[button] and not(last[button]) end -- Inner product of two vectors. If not the same length, just uses the shorter of the two. To capture the most recent values, multiplies only the last elements. local function inner(v1, v2) local prod = {} for i=0,math.min(#v1, #v2)-1 do table.insert(prod, v1[#v1-i]*v2[#v2-i]) end return sum(prod) end -- Updates the mouse's history. local function updatemouse(history) local curr=input.getmouse() table.remove(history, 1) table.insert(history, curr.X) return history end -- Returns the difference between adjacent x values, effectively the mouse velocity. local function getdeltas(history) local delta_x = {} for i=1,#history-1 do table.insert(delta_x, history[i+1]-history[i]) end return delta_x end -- Returns a sort of weighted average of the mouse's movement based on the window function. local function windowavg(v, window) local denom = sum(window) local num = inner(v, window) return num/denom end -- Makes a window function. Exponential with time. local function makewindow(length, lambda) local window={} for i=1,length do table.insert(window, math.exp(lambda*(i-length))) end return window end -- Updates the joypad input based on the threshold and its history. Also updates and returns the history. local function gameinput(threshold, mouseavg, joyhist, joyavg) local curr=false if math.abs(mouseavg)>threshold*(1+math.abs(joyavg)) then -- "1+math.abs(joyavg)" is just a guess. It maybe needs to be refined. curr=true end local lorr = sign(mouseavg) local buttons = {"Right", [-1]="Left"} if curr then joypad.set({[buttons[lorr]]=curr}) end table.remove(joyhist, 1) table.insert(joyhist, curr and lorr or 0) return joyhist end -- Adjusts a single parameter according to whether we press some keys. local function adjustparam(param, keys, last, downkey, upkey, mini, maxi, interval, eps) eps=eps or 0.0001 if justpressed(keys, last, upkey) then param = (param>=maxi-eps) and maxi or param+interval elseif justpressed(keys, last, downkey) then param = (param<=mini+eps) and mini or param-interval end return param end local function updatevalues(last, mouselambda, joylambda, length, threshold, debugmode) local keys = input.get() mouselambda = adjustparam(mouselambda, keys, last, "Number6", "Number7", 0, 1, 0.1) joylambda = adjustparam(joylambda, keys, last, "Y", "U", 0, 1, 0.1) length = adjustparam(length, keys, last, "H", "J", 2, 10, 1) threshold = adjustparam(threshold, keys, last, "N", "M", 10, 50, 2) if justpressed(keys, last, "B") then gui.clearGraphics() debugmode = not(debugmode) end return keys, mouselambda, joylambda, length, threshold, debugmode end -- Initialization local last = input.get() local mouselambda = 0.2 local joylambda = 0.2 local length = 5 local threshold = 20 local mousestart = input.getmouse() mousestart=mousestart.X local mousehist, joyhist = {}, {} local debugmode = false for i=1,10 do -- Keep a history of the last 10 mouse locations and joypad buttons. Changing length only changes the window. table.insert(mousehist, mousestart) table.insert(joyhist, 0) end while true do mousehist = updatemouse(mousehist) delta_x = getdeltas(mousehist) mousewindow = makewindow(length-1, mouselambda) mouseavg = windowavg(delta_x, mousewindow) joywindow = makewindow(length, joylambda) -- Note that the joypad window and mouse window are different lengths. I assume this either doesn't matter or makes very little difference. joyavg = windowavg(joyhist, joywindow) joyhist = gameinput(threshold, mouseavg, joyhist, joyavg) emu.frameadvance() keys, mouselambda, joylambda, length, threshold, debugmode = updatevalues(keys, mouselambda, joylambda, length, threshold, debugmode) -- Print all the parameters and history for troubleshooting purposes. if debugmode then gui.drawRectangle(0,0,240,160,"gray","gray") gui.pixelText(180,20,mouselambda) gui.pixelText(180,28,joylambda) gui.pixelText(180,36,length) gui.pixelText(180,44,threshold) for i=1,length-1 do gui.pixelText(180,44+8*i,delta_x[i]) gui.pixelText(200,44+8*i,joyhist[i]) gui.pixelText(220,44+8*i,mousehist[i]) end gui.pixelText(200,44+8*length,joyhist[length]) gui.pixelText(220,44+8*length,mousehist[length]) end end
I've briefly run this in debug mode to verify that it is at least mostly working, but you'll want to tweak the parameters yourself. There are four main parameters: mouselambda, joylambda, length, and threshold.
  • mouselambda affects how much we weight old mouse movements. Its default is 0.2. Set it to 0 and all mouse movements are equal. Its maximum value is 1, making it heavily weighted toward new movements. (Behind the scenes, each change in x is weighted by e^(-lambda) times the more recent change in x, so if lambda=1, the most recent delta-x gets a weight of 1 and the second most recent gets a weight of just 0.368, quickly going to zero.) To raise or lower its value, press 7 or 6, respectively.
  • joylambda is much like mouselambda, deciding how much we weight toward recent joypad inputs. Its default is 0.2. Set it to 0 and all joypad presses are equal. Its maximum value is 1, producing the same weighting strongly toward new joypad movements in exactly the same way as the mouselambda parameter. To raise or lower its value, press u or y, respectively.
  • length is how far back into the history we look. I keep the last ten values of each but truncate the window to this length. Its default is 5. Set it to 2 and only the most recent mouse movement matters (and if it's working correctly, the two lambda parameters are then irrelevant). Its maximum value is 10, which might cause a noticeable time lag, especially if your lambdas are small. To raise or lower its value, press j or h, respectively.
  • threshold is basically how far you have to move the mouse over the course of one frame in order for the script to register a left or right button press. This is the parameter I'm least certain about. I've allowed it to range from 10 to 50, with 10 being very sensitive to mouse movements and 50 being very insensitive. Its default is 20 and you can raise or lower its value in increments of 2 by pressing m or n, respectively. I also made a guess as to how it might be implemented, with the mouse movement needing to exceed the threshold plus the weighted average of previous joypad presses.
  • Additionally, I've included a "debug mode", which can be accessed by pressing b. This displays all parameters and recent mouse movements, joypad buttons, and mouse positions.
Very basically, setting mouselambda=0, joylambda=0, length=10, and threshold=50 makes input weak and laggy. Setting mouselambda=1, joylambda=1, length=2, and threshold=10 makes input strong and twitchy. Having said all that, I have an assignment for you: Run the script with the game of your choice (Revolution X?) and freely change the parameters by pressing 7, 6, u, y, j, h, m, and n until it's working to your satisfaction. Then switch to debug mode and tell me what the parameters are so I can change the defaults, ranges, and intervals. If there are any basic issues with the program, such as wanting more or less maximum sensitivity, more fine tuning, or a longer history, I can make those changes easily so don't be afraid to tell me about them! If there are more complex issues, please bring them to my attention anyway and maybe I'll discover there's something simple wrong with my assumptions or implementation. But in general, no promises. If we're lucky, the game accepts input frame-by-frame, rather than in pairs or with acceleration or something else that might affect how mouse movements translate to joypad movements. If that ends up being the case, troubleshooting my script will be rather difficult. Hope this works right out of the box! Edit: I've updated the threshold parameter to range from 10 to 50 in intervals of 2 because I perceived it wasn't sensitive enough. I also overhauled the updatevalues function, consolidating all the if statements into a new adjustparam function. Edit 2: I probably should have looked up a video of the game before starting this. For some reason, I assumed Revolution X was a top-down shoot-'em-up with the ship fixed at the bottom of the screen. As such, I only programmed in motion for the x axis. Oops. Adding in y axis functionality probably won't take all that long, but I kind of want to take a break so please be patient.
Player (80)
Joined: 8/5/2007
Posts: 865
New post because I don't want to keep updating my old one and my new version of the script is relatively inelegant. But here's y-axis support: Download Mouse2Move.lua
Language: lua

-- The sign of a scalar. local function sign(x) return (x>0) and 1 or ((x<0) and -1 or 0) end -- Sums over a vector. local function sum(v) local total=0 for i=1,#v do total=total+v[i] end return total end -- Did we just press this key? local function justpressed(keys, last, button) return keys[button] and not(last[button]) end -- Inner product of two vectors. If not the same length, just uses the shorter of the two. To capture the most recent values, multiplies only the last elements. local function inner(v1, v2) local prod = {} for i=0,math.min(#v1, #v2)-1 do table.insert(prod, v1[#v1-i]*v2[#v2-i]) end return sum(prod) end -- Updates the mouse's history. local function updatemouse(history, axis) local curr=input.getmouse() table.remove(history, 1) table.insert(history, curr[axis]) return history end -- Returns the difference between adjacent x values, effectively the mouse velocity. local function getdeltas(history) local delta_x = {} for i=1,#history-1 do table.insert(delta_x, history[i+1]-history[i]) end return delta_x end -- Returns a sort of weighted average of the mouse's movement based on the window function. local function windowavg(v, window) local denom = sum(window) local num = inner(v, window) return num/denom end -- Makes a window function. Exponential with time. local function makewindow(length, lambda) local window={} for i=1,length do table.insert(window, math.exp(lambda*(i-length))) end return window end -- Updates the joypad input based on the threshold and its history. Also updates and returns the history. local function gameinput(threshold, mouseavg, joyhist, joyavg, buttons) local curr=false if math.abs(mouseavg)>threshold*(1+math.abs(joyavg)) then -- "1+math.abs(joyavg)" is just a guess. It maybe needs to be refined. curr=true end local lorr = sign(mouseavg) if curr then joypad.set({[buttons[lorr]]=curr}) end table.remove(joyhist, 1) table.insert(joyhist, curr and lorr or 0) return joyhist end -- Adjusts a single parameter according to whether we press some keys. local function adjustparam(param, keys, last, downkey, upkey, mini, maxi, interval, eps) eps=eps or 0.0001 if justpressed(keys, last, upkey) then param = (param>=maxi-eps) and maxi or param+interval elseif justpressed(keys, last, downkey) then param = (param<=mini+eps) and mini or param-interval end return param end local function updatevalues(last, mouselambda, joylambda, length, threshold, debugmode) local keys = input.get() mouselambda = adjustparam(mouselambda, keys, last, "Number6", "Number7", 0, 1, 0.1) joylambda = adjustparam(joylambda, keys, last, "Y", "U", 0, 1, 0.1) length = adjustparam(length, keys, last, "H", "J", 2, 10, 1) threshold = adjustparam(threshold, keys, last, "N", "M", 10, 50, 2) if justpressed(keys, last, "B") then gui.clearGraphics() debugmode = not(debugmode) end return keys, mouselambda, joylambda, length, threshold, debugmode end -- Initialization local last = input.get() local mouselambda = 0.2 local joylambda = 0.2 local length = 5 local threshold = 20 local mousestart = input.getmouse() local mousexhist, mouseyhist, joyxhist, joyyhist = {}, {}, {}, {} local buttonsx = {"Right", [-1]="Left"} local buttonsy = {"Down", [-1]="Up"} -- Inverted because the y coordinate is read from the top of the screen. local debugmode = false for i=1,10 do -- Keep a history of the last 10 mouse locations and joypad buttons. Changing length only changes the window. table.insert(mousexhist, mousestart.X) table.insert(mouseyhist, mousestart.Y) table.insert(joyxhist, 0) table.insert(joyyhist, 0) end while true do mousexhist = updatemouse(mousexhist, "X") mouseyhist = updatemouse(mouseyhist, "Y") delta_x = getdeltas(mousexhist) delta_y = getdeltas(mouseyhist) mousewindow = makewindow(length-1, mouselambda) local mousexavg = windowavg(delta_x, mousewindow) local mouseyavg = windowavg(delta_y, mousewindow) local joywindow = makewindow(length, joylambda) -- Note that the joypad window and mouse window are different lengths. I assume this either doesn't matter or makes very little difference. local joyxavg = windowavg(joyxhist, joywindow) local joyyavg = windowavg(joyyhist, joywindow) joyxhist = gameinput(threshold, mousexavg, joyxhist, joyxavg, buttonsx) joyyhist = gameinput(threshold, mouseyavg, joyyhist, joyyavg, buttonsy) emu.frameadvance() keys, mouselambda, joylambda, length, threshold, debugmode = updatevalues(keys, mouselambda, joylambda, length, threshold, debugmode) -- Print all the parameters and history for troubleshooting purposes. if debugmode then gui.drawRectangle(0,0,240,160,"gray","gray") gui.pixelText(180,20,mouselambda) gui.pixelText(180,28,joylambda) gui.pixelText(180,36,length) gui.pixelText(180,44,threshold) for i=1,length-1 do gui.pixelText(180,44+8*i,delta_x[i]) gui.pixelText(20,44+8*i,delta_y[i]) gui.pixelText(200,44+8*i,joyxhist[i]) gui.pixelText(40,44+8*i,joyyhist[i]) gui.pixelText(220,44+8*i,mousexhist[i]) gui.pixelText(60,44+8*i,mouseyhist[i]) end gui.pixelText(200,44+8*length,joyxhist[length]) gui.pixelText(40,44+8*length,joyyhist[length]) gui.pixelText(220,44+8*length,mousexhist[length]) gui.pixelText(60,44+8*length,mouseyhist[length]) end end
Basically, I just copied all of the variables line-by-line. What I perhaps should have done instead was put both x and y values into a single table but... I didn't feel like it. Same request as above: run the script and tweak the parameters to your liking, then let me know what I should change.
RGL
Joined: 7/13/2017
Posts: 57
Damn... forgot to check this thread. Thank you dude. Ill try it out :)
RGL
Joined: 7/13/2017
Posts: 57
sorry if I am just too stupid. I can see how it works in the Debug mode and also think I get how all values work.. but, the game doesnt get any inputs. Do I have to check something first so that a Lua script is allowed to make inputs or something like that? Ive never used any script before :-/
Player (80)
Joined: 8/5/2007
Posts: 865
RGL wrote:
sorry if I am just too stupid. I can see how it works in the Debug mode and also think I get how all values work.. but, the game doesnt get any inputs. Do I have to check something first so that a Lua script is allowed to make inputs or something like that? Ive never used any script before :-/
You're not the only person who forgot to check this thread. I tested the script out in a game and it worked fine for me. Start by setting the sensitivity to the highest levels and moving the mouse vigorously to produce a constant input. If you get nothing out of it, then there's some problem with the script. Otherwise, you'll also want to check the game. Frame advance and alternate inputs of right and nil, which is commonly produced by my script. If the game does nothing, then it requires inputs in pairs, which I'm not really prepared to handle, especially since it will naturally lengthen the vector and make input laggier. If the crosshairs still move, then it again points to a problem in my script. Edit: You might also check whether the script works on other games. Sometimes there's problems with specific emulators.
RGL
Joined: 7/13/2017
Posts: 57
Well, I must be doing something wrong then. I tested several games and simply get no input at all. Open a game - open Lua Console - load mouse2move script ...anything else I need to do? I can see on the Debugscreen that it produces inputs but no game receieves any. "Display Input" doesnt show anything. TAStudio doesnt record any inputs either.
Player (80)
Joined: 8/5/2007
Posts: 865
RGL wrote:
Well, I must be doing something wrong then. I tested several games and simply get no input at all. Open a game - open Lua Console - load mouse2move script ...anything else I need to do? I can see on the Debugscreen that it produces inputs but no game receieves any. "Display Input" doesnt show anything. TAStudio doesnt record any inputs either.
Hrmm... that doesn't match my observations. I tested it with a Game Boy game as well as a GBA game and not only does it accept input, it also displays it on the screen with "Display Input". One thing to keep in mind is that your mouse movements are bound to the screen. If the mouse is already at the right edge of the screen, attempting to move further right will not work. This is a limitation of how mouse coordinates are read using Lua and I don't see any workaround; even if the script were working, it would require you to move fast in one direction and then slowly move back toward the center. Actually, I take it back. As long as you aren't doing anything that requires one-to-one mapping between the mouse and cursor, it shouldn't be especially difficult to program it such that your mouse position indicates the cursor's destination and inputs are automated such that you wait for it to catch up. It's not ideal, but it's probably much closer to what you had in mind. So assuming you're still interested, could you give me the following information for the game(s) of your choice? (Still Revolution X?)
  • The RAM addresses for the x and y coordinates of the cursor. These should be easy to find with RAM Search.
  • The minimum and maximum allowable values for these x and y coordinates. Again, use RAM Search. I'll assume x and y increase left to right and top to bottom, as is conventional, but be sure to let me know if that's not the case.
  • The screen resolution of the system you're using. You can find this with gui.drawPixel or some other function.
  • Which button do you want to represent "fire" (or "action")?
If you can give me that information, I should be able to write up a new script that gets the job done and in much fewer lines to boot. Edit: Also, it just occurred to me that the issue you're seeing maybe has something to do with input being encoded for your emulator as "left" and "right" instead of "Left" and "Right", as are used in my script. I thought that BizHawk had made those inputs uniform across all emulators, but it was a problem in the past and I may have incorrectly assumed it had been standardized. You might be able to make those edits to the script yourself. Try inserting a line of print(joypad.get()) somewhere in the main loop at the bottom of the script to see what keys the emulator accepts as input.