Hi there, Happy New Year! I’ve downloaded the new blender benchmark, which per the website also has a CLI version. When I run it, the process seems interactive. Is there any way to batch the process? Where does it save the data if I don’t want to upload it?
There’s an option to dump it out to the console in json see the output of
benchmark-launcher-cli --help for general help
The Blender Open Data Benchmark launcher command line interface Usage: benchmark-launcher-cli [flags] benchmark-launcher-cli [command] Available Commands: authenticate Request a new authentication token benchmark Benchmark scenes blender Commands for managing Blender versions clear_cache Removes all downloaded assets devices List the devices which can be benchmarked help Help about any command interactive Run the launcher in interactive mode (the default command) scenes Commands for managing scenes Flags: --browser open the verification URL in the browser -h, --help help for benchmark-launcher-cli -v, --verbosity uint8 control verbosity (0 = no logs, 1 = error logs (default), 2 = debug logs, 3 = dump blender output) (default 1) --version version for benchmark-launcher-cli Use "benchmark-launcher-cli [command] --help" for more information about a command.
the json output is in the benchmark command
Benchmark scenes Usage: benchmark-launcher-cli benchmark [scenes, ...] [flags] Flags: -b, --blender-version string the Blender version to benchmark --device-name string the name of the device to benchmark --device-type string the type of the device to benchmark -h, --help help for benchmark --json dump json output to stdout --submit submit the benchmark to the Blender Open Data website Global Flags: -v, --verbosity uint8 control verbosity (0 = no logs, 1 = error logs (default), 2 = debug logs, 3 = dump blender output) (default 1)
Hi there, when I do this:
benchmark-launcher-cli benchmark bmw27 --blender-version 2.81a --device-type CPU --json --verbosity 3
I get some errors:
2020/01/10 01:41:41 metadata.go:13: Fetching Metadata: https://opendata.blender.org/benchmarks/metadata/ 2020/01/10 01:41:42 metadata.go:48: Successfully fetched metadata 2020/01/10 01:41:42 metadata.go:70: Getting current launcher. 2020/01/10 01:41:42 checksum.go:13: Calculating checksum of current launcher. 2020/01/10 01:41:42 checksum.go:32: Launcher checksum: 88e5b9f3da9abce857347217f8999b93a9ae2bec72f9c5c7f74d2476054ecf9b. ERROR: Blender version 2.81a seems to be broken. Consider running the 'clear_cache' command. ERROR: Blender version 2.81a with checksum 87355b0a81d48ea336948294b9da8670eaae73667fae028e9a64cbb4104ceea1 is broken: missing valid flag file
Is there something else I need to do?
that command seems to run flawlessly for me
I only unzipped the CLI version, should I have anything else installed on the machine?
EDIT: Okay, so it looks like I need to run the tool once and answer some prompts first, and then the command invocation above works. Is there any way to get this to work without doing the interactive CLI step?
Just to bump this. A CLI version needs to be true CLI.
The benchmark needs to be offered as a standalone, not a run-and-download.
It’d be nice if we could choose where to install it - but a true standalone fixes that need.
Edit: Also, we need the authentication requirement to be removed. I’m running the benchmark across multiple systems not connected to the internet, with no intention of uploading results (given that I have access to NDA systems). The GUI version of the benchmark doesn’t seem to have the same authentication requirement, so I’m scratching my head as to why the Windows CLI version does.
I’ve downloaded the source and will work on building/debugging this issue. No timeline yet. How are issues logged and assigned when it comes to Blender?
Task on Manifest, and then claim it using the comment box, then a task on Differential.
Logged as: https://developer.blender.org/T73421
I just did a sync and rebuild of the launcher, and I’m seeing something odd. When I try to run it, I get:
ERROR: fetching metadata failed: current launcher not supported
Any ideas why?
Just a follow up on this issue. It should be addressed with the latest version. Someone beat me to it. There’s an accompanying readme that explains how to use the command to get the version of blender, download the data you need, and run the benchmarks. Note that there’s an issue with running it on a machine with no internet access, so that’s something that needs looking into. I’ll start a new thread on that one.
EDIT: Topic logged here: Blender Benchmark CLI won't run if no access to Internet?