I’m using Debian 12, Ryzen 7 5700X processor, and Radeon HD 5450 graphics card. I have tried uninstalling and reinstalling VLC but it didn’t resolve the issue. Here’s an excerpt from the VLC’s log file:
glconv_vaapi_x11 error: vaDeriveImage: operation failed
main error: video output creation failed
main error: failed to create video output
avcodec info: Using G3DVL VDPAU Driver Shared Library version 1.0 for hardware decoding
How do I resolve this issue?
First, it never hurts to reboot. There could be some dumb state going on in your display server. Or kernel DRM. Or in some little bs microcontroller in the video card.
Next, read the arch wiki on hardware video acceleration. Contemplate the note(2) at the very bottom of the page and boggle at all the PPANAPAPPI acronyms bouncing around in there.
VLC has two major sides to its video settings, the (Video)output method and the (Input/Codecs)hardware-acceleration. You are on the VDPAU acceleration API, so give VAAPI a try for a bit. Remember you have to restart VLC before any change takes. VLC should be smart about choosing a good Automatic option, but it can’t do much about “looks like an API’s there, but it’s broke”.
Try mpv. Try VLC, but from Flatpak (which brings its own version of a lot of the acceleration libraries).