Adobe's Impressive AI Upscaling Project Makes Blurry Videos Look HD
(Wednesday April 24, 2024 @05:20PM (msmash)
from the moving-forward dept.)
- Reference: 0173629174
- News link: https://tech.slashdot.org/story/24/04/24/1836221/adobes-impressive-ai-upscaling-project-makes-blurry-videos-look-hd
- Source link:
Adobe researchers have developed a new generative AI model called VideoGigaGAN that can [1]upscale blurry videos at up to eight times their original resolution . From a report:
> Introduced in a paper published on April 18th, Adobe claims VideoGigaGAN is superior to other Video Super Resolution (VSR) methods as it can provide more fine-grained details without introducing any "AI weirdness" to the footage. In a nutshell, Generative Adversarial Networks (GANs) are effective for upscaling still images to a higher resolution, but struggle to do the same for video without introducing flickering and other unwanted artifacts. Other upscaling methods can avoid this, but the results aren't as sharp or detailed. VideoGigaGAN aims to provide the best of both worlds -- the higher image/video quality of GAN models, with fewer flickering or distortion issues across output frames. The company has provided several examples here that show its work in full resolution.
[1] https://www.theverge.com/2024/4/24/24138979/adobe-videogigagan-ai-video-upscaling-project-blurry-hd
> Introduced in a paper published on April 18th, Adobe claims VideoGigaGAN is superior to other Video Super Resolution (VSR) methods as it can provide more fine-grained details without introducing any "AI weirdness" to the footage. In a nutshell, Generative Adversarial Networks (GANs) are effective for upscaling still images to a higher resolution, but struggle to do the same for video without introducing flickering and other unwanted artifacts. Other upscaling methods can avoid this, but the results aren't as sharp or detailed. VideoGigaGAN aims to provide the best of both worlds -- the higher image/video quality of GAN models, with fewer flickering or distortion issues across output frames. The company has provided several examples here that show its work in full resolution.
[1] https://www.theverge.com/2024/4/24/24138979/adobe-videogigagan-ai-video-upscaling-project-blurry-hd
Includes adding things that are not there (Score:2)
by gweihir ( 88907 )
Visual hallucinations, essentially.
Re: (Score:3)
by ChatHuant ( 801522 )
> Visual hallucinations, essentially.
I think this is unnecessarily harsh. A more correct description of the process would be "informed guesses", rather than "visual hallucinations".
Of course, the information isn't there to begin with, so the AI has to make up new data that never existed in the original. But the whole reason for the exercise isn't to extract information from the image - like "enhancing" it until the face of the perpetrator is seen in a reflection. It's to create a more visually pleasant photo from one that's technically imperfe
Re: (Score:2)
by gweihir ( 88907 )
There is nothing "informed" about AI.
re: Things not there, Visual hallucinations (Score:2)
by Tablizer ( 95088 )
Happens to me when I take off my undies.
Solved! (Score:2)
by Tablizer ( 95088 )
Han shot first via his 3rd arm.
Solving many a crime (Score:2)
If the claim is true, I would imagine law enforcement may be quite busy solving a lot of cold cases as a result. Crime filmed in BlurryHD has ironically become the video surveillance standard regardless of camera tech.
Re: (Score:3)
It can only make some plausible extrapolations and it can make these less well than a competent human. All the current hype AI has brought us is texts that sound good but often are bullshit and now pictures that look sharp but are not. You know, better sounding or better looking crap. Still crap.
Re: Solving many a crime (Score:2)
Given that the Kyle Rittenhouse trial had video thrown out because they used "pinch-to-zoom" on an iPad to make a portion of the video larger, this is a LONG ways from court admissability.
Re: (Score:2)
> it can make these less well than a competent human.
Not true. An AI can unblur much better than a human, especially with video where the AI can look at multiple frames simultaneously.
> All the current hype AI has brought us is texts that sound good
If you believe that, you're not paying attention.
Re: (Score:2)
> I would imagine law enforcement may be quite busy solving a lot of cold cases as a result.
If the enhanced image leads to other evidence, they might crack some cold cases.
But the unblurred images are unlikely to be directly admissible as evidence in court.
An AI can unblur a video much better than a human because the AI can combine the pixels from multiple frames.
Even better if the AI also has a clear photo of the scene taken later by the police. By comparing the clear photo to the blurry photo, an AI can figure out the characteristics of the lens and sensor and apply them to the perp's face, hand
Re: (Score:3)
> If the enhanced image leads to other evidence, they might crack some cold cases.
As the person who has been responsible for responding to law enforcement video requests and occasionally (three times) testifying as to that process and their authenticity, it's exceptionally rare (never personally seen it) for CCTV footage alone to convict someone. Most of the time it leads the police to a suspect, usually because someone they know recognized them (not for nothing that the police frequently publish these videos/images) and then the idiot convicts themselves by talking to the cops (pro-tip
Re: (Score:3)
Of course not, the video looks HD because what could be called more or less "AI hallucinations" have been added to it. So what you see isn't real, it would never stand in court.
Re: (Score:2)
That's the problem. I bet this will be applied to law enforcement to "zoom and enhance" like all the best crime dramas do. But remember, the enhanced details aren't really there . They're merely plausible fiction based on the information that is available, they're not adding any new information. People will see blurry security camera footage of a crime. The enhanced version will show a weapon where there is none, or no weapon when one was used. Detail will be added to faces that will make them look like on
Re: (Score:2)
> Cleaning up old movies or TV shows, for instance.
And porn. I knew I saved all those early aughts videos for a reason. ;-)
Re: (Score:2)
> That's the problem. I bet this will be applied to law enforcement to "zoom and enhance" like all the best crime dramas do. But remember, the enhanced details aren't really there.
Actually... that's not strictly true. Back in the 90's I worked at a company that worked with a vendor who could essentially motion-track a video and use those motion vectors to do a few things like upscale a video because knowing the motion of the camera means knowing what to do with the sub-pixel information you're getting. (fundamentally this is similar to how a scanner or copier works.) You can also use it to remove motion blur, especially if 3d points from the video can be intuited.
Am I saying the