Hi there, Happy New Year! I’ve downloaded the new blender benchmark, which per the website also has a CLI version. When I run it, the process seems interactive. Is there any way to batch the process? Where does it save the data if I don’t want to upload it?
There’s an option to dump it out to the console in json see the output of
benchmark-launcher-cli --help for general help
The Blender Open Data Benchmark launcher command line interface
Usage:
benchmark-launcher-cli [flags]
benchmark-launcher-cli [command]
Available Commands:
authenticate Request a new authentication token
benchmark Benchmark scenes
blender Commands for managing Blender versions
clear_cache Removes all downloaded assets
devices List the devices which can be benchmarked
help Help about any command
interactive Run the launcher in interactive mode (the default command)
scenes Commands for managing scenes
Flags:
--browser open the verification URL in the browser
-h, --help help for benchmark-launcher-cli
-v, --verbosity uint8 control verbosity (0 = no logs, 1 = error logs (default), 2 = debug logs, 3 = dump blender output) (default 1)
--version version for benchmark-launcher-cli
Use "benchmark-launcher-cli [command] --help" for more information about a command.
the json output is in the benchmark command
Benchmark scenes
Usage:
benchmark-launcher-cli benchmark [scenes, ...] [flags]
Flags:
-b, --blender-version string the Blender version to benchmark
--device-name string the name of the device to benchmark
--device-type string the type of the device to benchmark
-h, --help help for benchmark
--json dump json output to stdout
--submit submit the benchmark to the Blender Open Data website
Global Flags:
-v, --verbosity uint8 control verbosity (0 = no logs, 1 = error logs (default), 2 = debug logs, 3 = dump blender output) (default 1)
Hi there, when I do this:
benchmark-launcher-cli benchmark bmw27 --blender-version 2.81a --device-type CPU --json --verbosity 3
I get some errors:
2020/01/10 01:41:41 metadata.go:13: Fetching Metadata: https://opendata.blender.org/benchmarks/metadata/
2020/01/10 01:41:42 metadata.go:48: Successfully fetched metadata
2020/01/10 01:41:42 metadata.go:70: Getting current launcher.
2020/01/10 01:41:42 checksum.go:13: Calculating checksum of current launcher.
2020/01/10 01:41:42 checksum.go:32: Launcher checksum: 88e5b9f3da9abce857347217f8999b93a9ae2bec72f9c5c7f74d2476054ecf9b.
ERROR: Blender version 2.81a seems to be broken. Consider running the 'clear_cache' command.
ERROR: Blender version 2.81a with checksum 87355b0a81d48ea336948294b9da8670eaae73667fae028e9a64cbb4104ceea1 is broken: missing valid flag file
Is there something else I need to do?
that command seems to run flawlessly for me
I only unzipped the CLI version, should I have anything else installed on the machine?
EDIT: Okay, so it looks like I need to run the tool once and answer some prompts first, and then the command invocation above works. Is there any way to get this to work without doing the interactive CLI step?
Just to bump this. A CLI version needs to be true CLI.
The benchmark needs to be offered as a standalone, not a run-and-download.
It’d be nice if we could choose where to install it - but a true standalone fixes that need.
Edit: Also, we need the authentication requirement to be removed. I’m running the benchmark across multiple systems not connected to the internet, with no intention of uploading results (given that I have access to NDA systems). The GUI version of the benchmark doesn’t seem to have the same authentication requirement, so I’m scratching my head as to why the Windows CLI version does.
I’ve downloaded the source and will work on building/debugging this issue. No timeline yet. How are issues logged and assigned when it comes to Blender?
Task on Manifest, and then claim it using the comment box, then a task on Differential.
https://wiki.blender.org/wiki/Process/Contributing_Code
Logged as: https://developer.blender.org/T73421
1 Like
I just did a sync and rebuild of the launcher, and I’m seeing something odd. When I try to run it, I get:
ERROR: fetching metadata failed: current launcher not supported
Any ideas why?
Just a follow up on this issue. It should be addressed with the latest version. Someone beat me to it. There’s an accompanying readme that explains how to use the command to get the version of blender, download the data you need, and run the benchmarks. Note that there’s an issue with running it on a machine with no internet access, so that’s something that needs looking into. I’ll start a new thread on that one.
EDIT: Topic logged here: Blender Benchmark CLI won't run if no access to Internet?
Is there a way to write the total render time to a specific file?
You could redirect the output to a file and see it at the bottom, or run the render via Python and parse the output as it gets … outputted.
So Im currently running the blender benchmark cli through a subprocess.Popen() in my python script.
Im using the communicate() option to save the output. But for some reason, it can never grab the timing on the very last line of the output. Ive tried printing the output to the console to check it, and for some reason, it never captures all the output lines.
Ive also tried debugging by just running the command manually, and the render takes about 20 mins to complete. However, I noticed around the 18min mark, it just stops running. Please help.
This is what I do:
command = [BLENDER_LOCATION,
"--background",
"--factory-startup",
"-noaudio",
"--debug-cycles",
"--enable-autoexec",
"--engine", "CYCLES",
final_file_name]
command.extend(["--render-format", "PNG", "--render-output", "//{}_render_".format(currFile), "--render-frame", "1"])
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=False)
# Keep reading status while Blender is alive.
while True:
line = process.stdout.readline()
if line == b"" and process.poll() is not None:
break
line = line.decode().strip()
if line == "":
continue
Mind you I time the whole thing, but this does grab the last line, at least for me. The source for the blender benchmark is also a decent resource (it does this too if I remember correctly). Does that help?
Does anyone know how to do OPTIX+CPU type hybrid combo with the CLI version?
Also I get a long list of devices with the GUI version which is a bit confusing since it lists the CPU repeated times.
