PugetBench for Creators User Guide

This user guide includes details about how to use PugetBench command-line tools as well as our data collection policy, update logs, and more.

Download PugetBench for Creators

Quickly Jump To: Data Collection • Troubleshooting • CLI Guide • Sharing Assets • Update Log • Known Issues • Gallery

PugetBench for Creators is designed to test a range of Content Creation workflows and is ideal for end users, hardware reviewers, press, system integrators, and hardware manufacturers who want to examine the performance of different hardware in real-world workflows. Our benchmarks are easy to install and run, and have licensing options to support command-line automation and help prevent uploading NDA or confidential results.

Each benchmark has a low run-to-run margin of error and is highly repeatable. Results are presented as an Overall Score, as well as a breakdown of performance for individual tests and groups of tests.

Data Collection

When uploading a result to our public database, basic information on hardware specifications and application versions is recorded. All data is completely anonymous, and no personally identifiable information is collected. An example set of raw data is available below:

{
  "results": [
    {
      "test": "Overall Score (Standard)",
      "result": "100",
      "major": 2,
      "setting": "",
      "unit": "",
      "preset": "Standard"
    },
-----Repeated test results truncated-----
  ],
  "benchmark_version_major": 1,
  "benchmark_version_minor": "0.0",
  "benchmark_version_patch": 0,
  "benchmark_version_identifier": "",
  "application_version_major": 24,
  "application_version_minor": 0,
  "benchmark_name": "PugetBench for Premiere Pro",
  "application_name": "Premiere Pro",
  "specs_formatted": [
    {
    "type": "CPU",
    "manufacturer": "AuthenticAMD",
    "name": "AMD Ryzen 7 7700X 8-Core Processor",
    "cores": 8
  },
  {
    "type": "GPU",
    "manufacturer": "Advanced Micro Devices, Inc.",
    "name": "AMD Radeon Graphics",
    "driver": "31.0.23013.1023"
  },
  {
    "type": "GPU",
    "manufacturer": "NVIDIA",
    "name": "NVIDIA GeForce RTX 4080",
    "driver": "31.0.15.2824"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "OS",
    "manufacturer": "Microsoft Corporation",
    "name": "Microsoft Windows 11 Pro",
    "version": "22621"
  },
  {
    "type": "System",
    "manufacturer": "ASUS",
    "name": "System Product Name"
  },
  {
    "type": "Motherboard",
    "manufacturer": "ASUSTeK COMPUTER INC.",
    "name": "ProArt X670E-CREATOR WIFI",
    "bios": "1602"
  },
  {
    "type": "Engine",
    "name": "Mercury Playback Engine",
    "value": "CUDA"
  }
  ]
}

Troubleshooting

If you have any issues with the benchmark not properly running, first verify that your system meets the System Requirements (Photoshop | Premiere Pro). After verifying that your system and configuration meet the requirements, there are a number of troubleshooting steps you can try:

• Check the list of Known Issues
• If no application is shown in the drop-down, you most likely do not have a version installed that is supported by the benchmark
• Do not attempt to use the system while the benchmark is running. Defocusing the application window can break the benchmark in many situations.
• Ensure that your OS and application you are testing were installed in English.
• Disable Windows Defender or any real-time virus scanning programs.
• Clear the application preferences
  • Adobe: Follow Adobe’s guides (Photoshop | Premiere Pro) or use our Adobe CC Preference Backup/Restore/Clear utility (Windows only).
  • DaVinci Resolve: Deleting the “DaVinci Resolve” folder in “%appdata%\Blackmagic Design”
• Delete the benchmark assets, then re-download. These assets are stored in the following folder:
  • Windows: C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\assets
  • MacOS: ~/Library/Application Support/com.puget.benchmark/assets
• Photoshop
  • If you are getting an error about a PSD being too large, check that “Disable Compression of PSD and PSB Files” is unchecked in “Edit -> Preferences -> File Handling”

Lower-than-expected performance can be caused by a number of factors, but the most common are:

• Unknowingly comparing scores to different benchmark and/or application versions.
• Outdated BIOS/drivers/software
• Running background processes (Antivirus, Windows Updates, etc.)
• Windows power profile (High Performance is recommended for the best results)
• Adobe application settings are not correct (GPU acceleration not enabled, etc.)
• Corrupt application preferences (semi-common after changing GPU brands)

Benchmarks can also fail due to failing hardware, OS corruption, and other system instability issues.

If you have gone through the above steps and are still having issues, you are welcome to fill out a bug report. Please note that official support is only available if you have purchased a benchmark license. Be sure to include a copy of any log files that are generated by the benchmark and, ideally, a screenshot of the issue in question. Log files can be found in the following locations:

Windows:

• C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\logs
• C:\Users\[USERNAME]\AppData\Roaming\com.puget.benchmark\logs

MacOS:

• ~/Library/Application Support/com.puget.benchmark/logs
• ~/Library/Logs/com.puget.benchmark

Command Line (CLI) Guide

The PugetBench for Creators application is installed by default in the following locations:

• Windows: C:\Program Files\PugetBench for Creators\PugetBench for Creators.exe
• MacOS: /Applications/PugetBench for Creators.app/Contents/MacOS/PugetBench for Creators

Common CLI Arguments

Here are the most common command line parameters for PugetBench:

--help

--license

--app

--app_version

--benchmark_version

--preset

--upload_results

CLI Examples

Activate your benchmark license

--license "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX"

Run the 1.0.0 Premiere Pro Standard benchmark on Premiere Pro 23.5 and upload the results

--app "Premiere Pro" --app_version 23.5 --benchmark_version 1.0.0 --preset "Standard" --upload_results

Run the 1.0.0 Photoshop Standard benchmark on Photoshop 24.0.0 with 2 loops, retrying 3 times in the case of an error/application crash, and copy the results to the current folder (Windows)

--app photoshop --app_version 24.0.0 --benchmark_version 1.0.0 --preset "Standard" --run_count 2 --rerun_count 3 --copy_log "%~dp0copy_log.csv"

Sharing Assets Between Systems

If you are utilizing the CLI functionality, you may also want to avoid needing to download the benchmark assets onto multiple systems. To do this, follow these steps:

1. On one system, install the PugetBench for Creators application and download the assets for each benchmark you plan to run.
2. Open the benchmark log folder by going to “Settings”, and clicking on “Reveal Logs”
3. Copy the “assets” and “benchmarks” to an external drive, NAS, etc.
4. These two folders can be copied to any other system with the PugetBench for Creators application installed. They are also Windows/MacOS agnostic, although the path will be different for each OS.

Application Update Log

Update log for the desktop app. Updates for individual benchmarks are recorded on each benchmark’s information page.

Version 1.2.22

• Adjusted scoring for increased accuracy. These changes should maintain Overall Scores within the margin of error while improving precision, particularly for systems with lower results.
  • Overall Scores are now calculated from the original, unrounded sub-score results, such as “134.643256482” rather than “135.”
  • Sub-scores are now rounded to a fixed number of decimal places when logged, ensuring that each has at least 3 significant digits. Examples:
    • 5 ➝ 5.23
    • 68 ➝ 67.6
    • 135 ➝ 135 (no change)
    • 2685 ➝ 2685 (no change)
  • Please note that the lower the sub-score, the larger the run-to-run variation tends to be. While the rounding adjustment will improve accuracy, we recommend running multiple benchmark passes on lower-powered systems and using the average when possible.

Version 1.2.21

• Added support for DaVinci Resolve (Studio) 19

Version 1.2.20

• Added support for PugetBench for DaVinci Resolve.
• Added helper links alongside the “Benchmark Preset” and “Benchmark Version” dropdowns to take the user to the relevant page explaining the differences between the available options.
• Added a Creators Application Version and Benchmark Version compatibility checker to ensure that the selected benchmark will work properly with the installed Creators application.
• Fixed a bug with Windows specs gathering that would cause the benchmark to hang if a class doesn’t exist.
• Added “–hardware_encoding_mode” CLI argument for DaVinci Resolve to allow the manual selection of the H.264/HEVC encoding mode (AMD/Intel/NVIDIA/Metal)
• If local .csv logging is enabled via the GUI or CLI, do not open the online results page if result uploading is enabled.
• Misc. bug fixes.

Version 1.1.7

• Bug fix for activating benchmark licenses via CLI using the –license argument

Version 1.1.5

• Back-end overhaul of system spec formatting.
  • This will make adding new specs easier in the future and help us filter NDA results.
  • The format change will also resolve some existing bugs from unusual hardware configurations.
• Added client-side blacklist for common early NDA/pre-release hardware values.
  • If pre-release hardware is detected, a banner is shown and result uploading is disabled.
  • The Overall Score is still displayed in the UI after a benchmark run.
  • Full results can still be logged to a local .csv file when using a license.
• Offline activation and lockdown modes added for enterprise users.
  • For enterprise (site license) users only.
  • Includes offline activation and the ability to lock result uploading and/or benchmark downloading in the UI.
  • Even if uploading is blocked in the UI, running the benchmark via the CLI with the –upload_results argument will override the setting and upload results.
• Added a notification banner to notify the user when a newer version of the PugetBench for Creators application is available.
• CLI updates
  • –app_version now only requires the major application version. I.E. “24” rather than “24.2.0”. Using more specific version numbers is still supported.
  • –app_version and –benchmark_version arguments are now optional. If not filled in, the benchmark will default to using the latest version found on the system. Note that if you have beta builds of Photoshop or Premiere Pro installed, it will attempt to use those as they are newer than the release versions. However, beta builds are not officially supported and may have issues.
• Bug Fixes
  • Fixed several issues with system spec gathering that could cause the benchmark to halt before generating the final score.
  • The Adobe extensions folder will now be generated automatically if it is missing. This resolves a bug on MacOS for users who have never installed an Adobe plugin.
  • Fixed Premiere Pro beta detection. However, beta versions are not officially supported since Adobe may make changes that break the benchmark.
  • Miscellaneous UI and back-end bug fixes.

Version 1.0.5

• Fix for low results in Photoshop benchmark due to improper logging of the fastest result across multiple runs.
• Fix for Premiere Pro benchmark sometimes not generating scores.

Version 1.0.2

• Bug fix to allow non-licensed application versions to run the “Extended” Premiere Pro benchmark preset.

Version 1.0.1

• Fix for an intermittent asset download issue.
• Fix to allow non-licensed users to run Premiere Pro Extended preset.

Version 1.0.0

• Initial Release
• Support for Premiere Pro and Photoshop. Other benchmarks are coming soon!
• Individual benchmark licenses are no longer supported. If you have an app-specific license, please contact [email protected] for a free upgrade.

Known Issues

Known issues and possible workarounds for the latest version of PugetBench for Creators are listed below:

• “Error compiling movie” error when running the ARRIRAW processing test on systems with multiple NVIDIA GPUs. This is a Premiere Pro bug in version 23.1 and newer. The workaround until Adobe releases a fix is to disable or remove all but the primary GPU.

Gallery

Previous Next

System Image

×

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Previous Next