hi nanogyth,
since I'm interested in speedruns and also avisynth and video editing I've come to see this thread and your script. I must say, pretty amazing work there.
But I noticed that the detection of shakes and global flashes is not working pretty well (well that's probably the reason you put it in test mode).
I came up with a pretty good solution for this problem. basically it doesn't use the local motion (like you did with mmask) but the global motion compensation aka the camera pan or scrolling.
here is the function for 1frame-flashes:
function sfx_flash(clip clp, int "thr"){
thr = default(thr, 64)
src = clp.ConvertToYV12()
super = MSuper(src, pel=1)
fvec = MAnalyse(super, isb=false, blksize=4)
gmc = MDepan(src, fvec)
fgmc = Depan(src, data=gmc, offset=-1)
bgmc = Depan(src, data=gmc, offset=1)
fflash = mt_lutxy(fgmc,src,expr="x y - abs 2 *").mt_binarize(thr)
bflash = mt_lutxy(bgmc,src,expr="x y - abs 2 *").mt_binarize(thr)
flash = mt_logic(fflash,bflash,mode="and")
mt_lutf(flash,flash,"avg",expr="x").mt_binarize(thr).GreyScale()
}
litte explanation: let's say (c)urrent is our reference frame, (f)orward and (b)ackward the neighbor frames and diff(x,y) the difference between frames x and y. we eliminate the scrolling effect (which distorts the difference calculation) by analyzing the global motion (with MDepan() and Depan()) and shifting the frame to the postion of the reference frame.
the rest is simple: we move f and b with depan to the position of c. if c is a single frame flash then diff(c,f) and diff(c,b) is pretty big and we set the mask to 255. otherwise it's zero. the threshold maybe needs adjustment from game to game.
and now the code for 1frame-shakes:
function sfx_shake(clip clp, int "thr"){
thr = default(thr, 32)
src = clp.ConvertToYV12()
super = MSuper(src, pel=1)
fvec = MAnalyse(super, isb=false, blksize=4)
gmc = MDepan(src, fvec)
b1gmc = Depan(src, data=gmc, offset=1)
b2gmc = Depan(src, data=gmc, offset=2)
b1shake = mt_lutxy(src.selectevery(1,-1),b1gmc,expr="x y - abs 2 *").mt_binarize()
b2shake = mt_lutxy(src.selectevery(1,-1),b2gmc.selectevery(1,1),expr="x y - abs 2 *").mt_binarize()
b1shkavg = mt_lutf(b1shake,b1shake,"avg",expr="x")
b2shkavg = mt_lutf(b2shake,b2shake,"avg",expr="x")
mt_lutxy(b1shkavg,b2shkavg,"x y -").mt_binarize(thr).GreyScale()
}
explanation: if c is a single frame flash then diff(b,c) should be greater than diff(b,f) otherwise the camera is standing still or we have normal motion. to compensate for the differences inside the actual frames we don't compare the given frames but again using the global motion analysis.
we shift b to the position of c and compare to the original position of b. then we do the same with b and f. the rest is just building the differences and comparing the results.
the problem I have now is that I don't really now how I use the information of these masks, especially when there are many flashes/shakes in a row.
here is my code so far, which works good if the source only has single flashes/shakes. maybe you have some ideas (should've add parameters for the new threshold values):
function sfx_deblinkmod(clip clp,
\ float "ratio",
\ int "level",
\ clip "blinkmask"
\){
deblink = ng_deblink(clp, ratio, level, blinkmask)
sfx = mt_lutxy(sfx_flash(deblink),sfx_shake(deblink),"x y +").ConvertToRGB32()
f0=Layer(deblink.SelectEvery(2,0),
\ deblink.SelectEvery(2,1).Mask(sfx.SelectEvery(2,1)),
\ level=257)
f1=Layer(deblink.SelectEvery(2,1),
\ deblink.SelectEvery(2,0).Mask(sfx.SelectEvery(2,0)),
\ level=257)
interleave(f0,f1)
}
you can easily add the 3 functions to your script and it should work. of course you can optimize the code by eliminating the additional msuper and manalyze calls. but this would have required a rewrite of a great part of your script which had made my changes incomprehensible.
tested the script modification with castlevania3, super metroid, bionic commando and contra3.
