Puget Systems Benchmark Tool for Pix4D

Always look at the date when you read an article. Some of the content in this article is most likely out of date, as it was written on May 15, 2018. For newer information, see Puget Systems Benchmark Tool for Pix4D

Table of Contents

Introduction

Here at Puget Systems, one of our primary goals is to make sure that our customers end up with a fast, reliable workstation that is perfectly tailored to their unique workflow. The main way we do this is by benchmarking a wide range of hardware across popular software – then publishing the results and using them to create recommended systems for each application.

However, while this ensures we are selling the right hardware, it does not give our customers a great idea of how much faster a new workstation would be compared to their existing system. To address this issue for Pix4D, we have created a tool that walks users through the same sort of testing we do internally.

Depending on your browser, OS, and antivirus security settings, you may receive warnings about this tool being an unrecognized app. Rest assured that if you downloaded it from the Puget Systems website it is not harmful in any way.

Download from Google Drive
Puget Systems Pix4D Benchmark Tool
(Beta Version 2018.05)

How to run the benchmark

Simply download and then run the executable linked to above. It will pop up a series of windows with instructions about what image sets to download, where to find them, and what settings to use when setting up each project in Pix4D. Once the projects have been processed, the tool will analyze the log files and display the results. The log files and results are also saved for future reference.

Depending on the hardware in your system, the process of project creation and processing involved in this benchmark should take between 15 and 60 minutes… possibly even longer on old or low-spec systems. It is not recommended to use your computer for anything else during the benchmark process since other activity would slow down Pix4D and lead to longer calculation times.

Compatibility

This benchmark was designed with Pix4Dmapper Pro 4.2.25, on Windows 10, but it should work with future versions as well – at least as long as the 3D Map and 3D Model templates remain the same, along with the log file formatting. Comparing results across different versions could also be used to measure optimizations made to Pix4D itself, but would not be fair for comparing different hardware.

In order to follow along with the instructions in this tool, you will need to have Pix4Dmapper Pro installed and logged in. If you are new to Pix4D, you can sign up for a 15-day trial on their website which will allow you to run this benchmark.

This benchmark has no official support from either Puget Systems or Pix4D and simply is intended as a tool for users in the photogrammetry community. If you have questions, comments, or suggestions please feel free to post them below.

How to interpret the results

Our benchmark utility walks through the normal steps of creating and processing image sets with two of Pix4D's built-in templates: 3D Model and 3D Map. Fairly small photo sets are used, to keep the calculation times reasonable while still ensuring the results can be extrapolated to larger projects.

The several tasks involved in turning still images into 3D representations are combined by Pix4D into two steps for models or three steps for maps. This benchmark utility pulls the time for each step from the log file that Pix4D creates, displaying those results and recording them to a file for later use. Both the individual steps and the total time are useful when comparing the performance of different systems.

Times shown are in seconds, with lower numbers (shorter processing times) being better. We have not implemented any sort of scoring system at this point since it is fairly simple to just compare the times between different benchmark runs. This way we are also not artificially placing more or less weight on different steps. Please note that both the CPU and GPU in a system affect Pix4D performance.

Sample results for comparison

Here are sample results from workstations built here at Puget Systems in the first few weeks of our using this benchmark internally:

System Specs	Eagle Model Step 1	Eagle Model Step 2	Eagle Model Total	Building Map Step 1	Building Map Step 2	Building Map Step 3	Building Map Total
Core i7 8700K + GTX 1050 Ti 4GB	152	383	535	163	302	465	930
Core i7 8700K + GTX 1080 8GB	117	373	490	140	300	440	880
Core i9 7960X + GTX 1080 Ti 11GB	119	278	397	147	226	292	665
2 Xeon E5-2690v4 + Titan Xp 12GB	148	317	465	183	256	341	780