Add evp_signature perftool

[andrewkdinh]

Summary

Adds a CLI tool evp_signature that signs data using RSA and a SHA256 digest as input.
Runs for 5 seconds and prints the average execution time per computation.

Fixes openssl/project#1841

Features

• Two modes of operation:
  • evp_shared (default): Use EVP API and allow shared data between computations
  • evp_isolated: Use EVP API and don't allow shared data between computations
• Configurable number of times to update
• Terse output for easier CI automation (-t)
• Configurable thread count

TODO: Since freeze functionality hasn't been properly added yet (see openssl/project#1836), support for freezing the context store has not been added yet.

Usage

$ ./evp_signature -h
Usage: evp_signature [-h] [-t] [-o operation] [-V] thread-count
-h - print this help output
-t - terse output
-o operation - mode of operation. One of [evp_isolated, evp_shared] (default: evp_shared)
-V - print version information and exit
thread-count - number of threads

$ ./evp_rand -o evp_shared 10 # evp_shared operation mode, 10 threads
Average time per computation: 912.408759us

$ ./evp_rand -o evp_isolated 10 # now using evp_isolated mode
Average time per computation: 70721.357850us

Findings

Initial results showed that evp_isolated takes about 70x time as evp_shared, but I'm not sure if this is measuring exactly what we want to test for EVP_SIGNATURE

[mbroz]

Shouldn't be this controlled by a parameter? I think 3072/4096 bits RSA is quite common these days.
Anyway, that can be changed later once needed.

[jogme]

+1

[andrewkdinh]

If EVP_PKEY_CTX_set_rsa_keygen_bits isn't called, then by default the RSA key length for RSA key generation is set to 2048 bits. Anyways, I don't think having an extra parameter for this is too useful (especially if other algorithms are added), so I just removed the call to EVP_PKEY_CTX_set_rsa_keygen_bits.

https://docs.openssl.org/master/man3/EVP_PKEY_CTX_ctrl/#rsa-parameters

[jogme]

Makes me wonder if we don't want to test different bit sizes, then what is the actual motivation behind this perftool?

[npajkovsky]

We don't need to test different sizes because we're pref testing alg fetching.

[npajkovsky]

We don't need to test different sizes because we're pref testing alg fetching.

Do we? IMO it does not make any sense to perf-test alg fetching for signatures. The overhead of fetching from a provider (even without the frozen context) will be IMO negligible in comparison to the actual signature or verification operation.

I found the same thing with EVP_KEM and EVP_KEYMGMT.

I think this perftool might be potentially useful to test the performance of particular signature algorithm but then it should be more configurable. Having it just to test the alg fetching with or without frozen context does not make any sense. For that it it is IMO fully sufficient to have the EVP_MD and EVP_CIPHER tests.

I don't have a problem with having more EVP_* perf tests.

[jogme]

IMO it would be better to have a evp_fetch_speed or such tool, where all the options would be added such as KEM, KEYMGMT, SIGNATURE, etc. instead of crafting separate tests for each if they won't be used for anything else.

I also understand it that the real value of performance tests is testing algorithms speed (and detect anomalies).

+1

[npajkovsky]

Hmm, we already have that in openssl speed.

[bob-beck]

So this looks to be sort of a "repeat" of the Evp hash one, combined with a test for EVP_pkey - is it not andrew?

This is not meant to be a test of the algorithm itself, but rather to track the overhead of the various methods
we have put in place for people to actually get the point where they can make use of an algorithm

So does it make sense to do this test as effecively, a combination of what it takes to get a hash, and to get a pkey? or should we just have a pkey only test.

[andrewkdinh]

So does it make sense to do this test as effecively, a combination of what it takes to get a hash, and to get a pkey? or should we just have a pkey only test.

@bob-beck I'm not sure either. Does having a pkey only test help us test what we're looking for?

If it were just to be pkey only test, would it essentially just be this?

    if ((pctx = EVP_PKEY_CTX_new_from_name(NULL, "RSA", NULL)) == NULL
        || EVP_PKEY_keygen_init(pctx) <= 0
        || EVP_PKEY_keygen(pctx, &pkey) <= 0) {