• 0 Posts
  • 11 Comments
Joined 2 years ago
cake
Cake day: June 14th, 2023

help-circle
  • In contrast, and I say this as someone who has used various types of Unix and Linux for a long time, I think this is an excellent use for AI, just be sure to use it to teach you things not just to solve your problems for you.

    What I mean by this is I have found (mostly Claude) to be great at explaining concepts, especially if you use it to make analogies to something you know. It is absolutely not right every single time but I have had great luck with questions like “explain to me how to X in Y tool, I know how to have the same outcome by doing A in B tool” or “explain to me how docker works using a rocket as a metaphor” or things like that. Also I use it a lot for new subjects where I don’t know what to search for quite yet and I can just give it a long rambling explanation and example and ask it for 3 suggestions to research further or things to check. It is kind of useful as an expensive search engine but if you use it like a research engineer to get you started it can be really helpful in my experience.

    As others have said though, I have been doing it forever both personally and professionally and I am definitely still learning. Linux knowledge is more of a skill to develop over time not something that is easy to master because it continually changes. Learning how to find or figure out the answers is the most valuable skill though, it’s impossible to remember everything. That and often there is no single right or correct answer for every situation but there are a lot of options and opinions and often more of the latter than the former. That said though usually the best answer is the one that I forget about because it functions forever and doesn’t blow up in my face hah.

    Anyway, hope at least some of that is helpful, best of luck!

    :wq


  • I think you already decided what I would have recommended (just write to a log file in your python script) but I wanted to hopefully help with the rest of the question hah.

    So the first thing to remember is that a pipe (|) in Linux is a unidirectional data channel that passes stdout from the left command the right command’s stdin and this is its only function. Also notable is that exit status of a pipeline is the exit status of the last command in the pipeline (unless the pipefail option is enabled but this isn’t the behavior you wanted either), this is what is available in $? as well immediately after the pipe exits in a script, problem with that is that tee can exit successfully while the previous command failed because it did its job redirecting output.

    To get the behavior you are after you would probably need to write a script that does the signal handling or it might work if you use exec to wrap your python+tee command in your dockerfile because then the bash process will get replaced by python or tee, I’m not sure which or how tee will interact with exec without testing though.

    Anyway, hope that helps, here are the docs on pipe which are worth a read. In fact when double checking something just now, I learned I can do |& today instead of 2>&1 | which is neat hah!

    Edit: I forgot to mention, signal handing in docker is a whole other animal so depending on how you are specifically running it the behavior and signals might not be what is expected or the same as running the commands outside of docker.

    Great article about it: https://medium.com/@gchudnov/trapping-signals-in-docker-containers-7a57fdda7d86

    Repost if you can’t read it on medium: https://www.cloudbees.com/blog/trapping-signals-in-docker-containers


  • I had a few ideas, I’m suspicious that handbrake is falling back to CPU, maybe check the logs of the container to make sure it isn’t falling back to CPU decoding. Otherwise here are a few things I would check next:

    • If you are not using docker locally so you are already doing this, you will need to configure the docker container to pass through the GPU for quicksync to work inside the container.
    • If you are already doing that then I would make sure the device is the same name on the synology, it probably is but just to be sure.
    • you will likely need to add your user to the video and/or render group on the synology if you haven’t, especially if you are running the container as your user instead of root
    • make sure you are reading and writing to volumes that use bind mounts and not docker volumes, overlayfs is not what I would call fast and writing especially.

  • So I had a few thoughts. I’m not sure that you can use the docker device flag with a directory as you have there, I think it expects a device node, you can pass that directory as a volume (-v) though.

    If that doesn’t work you might also try running the VM with host-passthrough mode set on the CPU as well if it isn’t set that way already, sometimes that is also required for pass through to work from my experience. Also, make sure you passed through the whole device node, sometimes there are audio devices you have to pass through with the GPU device or you will get odd errors like those initialization ones you had. I’m not sure if this is the case for Intel iGPU though offhand though. Are you able to use intel_gpu_top on the VM to access the GPU? None of that is necessarily specific to proxmox though (but probably applies to anything libvirt powered) so YMMV.

    Edit: I realized you may not know what a “device node” is, that is the full path to the device, like /dev/dri/renderD128 vs /dev/dri which is actually a directory.



  • Oh it is certainly not just you, I am sometimes confused reading them even for commands I have used for years and I know what flag I am looking for but don’t remember the exact syntax or something hah! I am glad they are there but they are definitely not a complete guide to any command, especially built-ins.

    Interestingly, this is something AI has been very useful for to me, less searching because I can describe the outcome I want and it figures out what I am talking about generally.



  • Okay so when you say “unplug the power” do you mean shut it down first or just pull the plug? The latter is a great way to corrupt your storage pools as ZFS uses memory for read and write cache etc by default. You definitely need to do a graceful shutdown especially if there is data that was recently written to disk, that’s why a UPS is so recommended. That said you can usually import an existing pool when that happens, I think there is a UI menu for it now.