summaryrefslogtreecommitdiffstats
path: root/drivers/crypto/virtio
AgeCommit message (Collapse)AuthorLines
2026-02-21Convert more 'alloc_obj' cases to default GFP_KERNEL argumentsLinus Torvalds-2/+1
This converts some of the visually simpler cases that have been split over multiple lines. I only did the ones that are easy to verify the resulting diff by having just that final GFP_KERNEL argument on the next line. Somebody should probably do a proper coccinelle script for this, but for me the trivial script actually resulted in an assertion failure in the middle of the script. I probably had made it a bit _too_ trivial. So after fighting that far a while I decided to just do some of the syntactically simpler cases with variations of the previous 'sed' scripts. The more syntactically complex multi-line cases would mostly really want whitespace cleanup anyway. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21Convert 'alloc_obj' family to use the new default GFP_KERNEL argumentLinus Torvalds-6/+6
This was done entirely with mindless brute force, using git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' | xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/' to convert the new alloc_obj() users that had a simple GFP_KERNEL argument to just drop that argument. Note that due to the extreme simplicity of the scripting, any slightly more complex cases spread over multiple lines would not be triggered: they definitely exist, but this covers the vast bulk of the cases, and the resulting diff is also then easier to check automatically. For the same reason the 'flex' versions will be done as a separate conversion. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21treewide: Replace kmalloc with kmalloc_obj for non-scalar typesKees Cook-8/+8
This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook <kees@kernel.org>
2026-01-31crypto: virtio - Replace package id with numa node idBibo Mao-1/+1
With multiple virtio crypto devices supported with different NUMA nodes, when crypto session is created, it will search virtio crypto device with the same numa node of current CPU. Here API topology_physical_package_id() is replaced with cpu_to_node() since package id is physical concept, and one package id have multiple memory numa id. Signed-off-by: Bibo Mao <maobibo@loongson.cn> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2026-01-31crypto: virtio - Remove duplicated virtqueue_kick in ↵Bibo Mao-2/+0
virtio_crypto_skcipher_crypt_req With function virtio_crypto_skcipher_crypt_req(), there is already virtqueue_kick() call with spinlock held in function __virtio_crypto_skcipher_do_req(). Remove duplicated virtqueue_kick() function call here. Fixes: d79b5d0bbf2e ("crypto: virtio - support crypto engine framework") Cc: stable@vger.kernel.org Signed-off-by: Bibo Mao <maobibo@loongson.cn> Acked-by: Jason Wang <jasowang@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2026-01-31crypto: virtio - Add spinlock protection with virtqueue notificationBibo Mao-0/+5
When VM boots with one virtio-crypto PCI device and builtin backend, run openssl benchmark command with multiple processes, such as openssl speed -evp aes-128-cbc -engine afalg -seconds 10 -multi 32 openssl processes will hangup and there is error reported like this: virtio_crypto virtio0: dataq.0:id 3 is not a head! It seems that the data virtqueue need protection when it is handled for virtio done notification. If the spinlock protection is added in virtcrypto_done_task(), openssl benchmark with multiple processes works well. Fixes: fed93fb62e05 ("crypto: virtio - Handle dataq logic with tasklet") Cc: stable@vger.kernel.org Signed-off-by: Bibo Mao <maobibo@loongson.cn> Acked-by: Jason Wang <jasowang@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-18crypto: engine - remove request batching supportOvidiu Panait-1/+1
Remove request batching support from crypto_engine, as there are no drivers using this feature and it doesn't really work that well. Instead of doing batching based on backlog, a more optimal approach would be for the user to handle the batching (similar to how IPsec can hook into GSO to get 64K of data each time or how block encryption can use unit sizes much greater than 4K). Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com> Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-06-23crypto: virtio - Remove unused virtcrypto functionsDr. David Alan Gilbert-38/+0
virtcrypto_devmgr_get_first() and virtcrypto_dev_in_use() were added in 2016 by commit dbaf0624ffa5 ("crypto: add virtio-crypto driver") but have remained unused. Remove them. Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-03-15crypto: virtio - Erase some sensitive memory when it is freedChristophe JAILLET-4/+2
virtcrypto_clear_request() does the same as the code here, but uses kfree_sensitive() for one of the free operation. So, better safe than sorry, use virtcrypto_clear_request() directly to save a few lines of code and cleanly free the memory. Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Tested-by: Lei Yang <leiyang@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: virtio - Drop superfluous [as]kcipher_req pointerLukas Wunner-10/+7
The request context virtio_crypto_{akcipher,sym}_request contains a pointer to the [as]kcipher_request itself. The pointer is superfluous as it can be calculated with container_of(). Drop the superfluous pointer. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: virtio - Drop superfluous [as]kcipher_ctx pointerLukas Wunner-8/+5
The request context virtio_crypto_{akcipher,sym}_request contains a pointer to the transform context virtio_crypto_[as]kcipher_ctx. The pointer is superfluous as it can be calculated with the cheap crypto_akcipher_reqtfm() + akcipher_tfm_ctx() and crypto_skcipher_reqtfm() + crypto_skcipher_ctx() combos. Drop the superfluous pointer. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: virtio - Drop superfluous ctx->tfm backpointerLukas Wunner-9/+0
struct virtio_crypto_[as]kcipher_ctx contains a backpointer to struct crypto_[as]kcipher which is superfluous in two ways: First, it's not used anywhere. Second, the context is embedded into struct crypto_tfm, so one could just use container_of() to get from the context to crypto_tfm and from there to crypto_[as]kcipher. Drop the superfluous backpointer. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: virtio - Simplify RSA key size cachingLukas Wunner-11/+8
When setting a public or private RSA key, the integer n is cached in the transform context virtio_crypto_akcipher_ctx -- with the sole purpose of calculating the key size from it in virtio_crypto_rsa_max_size(). It looks like this was copy-pasted from crypto/rsa.c. Cache the key size directly instead of the integer n, thus simplifying the code and reducing the memory footprint. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22crypto: virtio - Fix kernel-doc of virtcrypto_dev_stop()Lukas Wunner-1/+1
It seems the kernel-doc of virtcrypto_dev_start() was copied verbatim to virtcrypto_dev_stop(). Fix it. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-10-05crypto: virtio - Drop sign/verify operationsLukas Wunner-44/+21
The virtio crypto driver exposes akcipher sign/verify operations in a user space ABI. This blocks removal of sign/verify from akcipher_alg. Herbert opines: "I would say that this is something that we can break. Breaking it is no different to running virtio on a host that does not support these algorithms. After all, a software implementation must always be present. I deliberately left akcipher out of crypto_user because the API is still in flux. We should not let virtio constrain ourselves." https://lore.kernel.org/all/ZtqoNAgcnXnrYhZZ@gondor.apana.org.au/ "I would remove virtio akcipher support in its entirety. This API was never meant to be exposed outside of the kernel." https://lore.kernel.org/all/Ztqql_gqgZiMW8zz@gondor.apana.org.au/ Drop sign/verify support from virtio crypto. There's no strong reason to also remove encrypt/decrypt support, so keep it. A key selling point of virtio crypto is to allow guest access to crypto accelerators on the host. So far the only akcipher algorithm supported by virtio crypto is RSA. Dropping sign/verify merely means that the PKCS#1 padding is now always generated or verified inside the guest, but the actual signature generation/verification (which is an RSA decrypt/encrypt operation) may still use an accelerator on the host. Generating or verifying the PKCS#1 padding is cheap, so a hardware accelerator won't be of much help there. Which begs the question whether virtio crypto support for sign/verify makes sense at all. It would make sense for the sign operation if the host has a security chip to store asymmetric private keys. But the kernel doesn't even have an asymmetric_key_subtype yet for hardware-based private keys. There's at least one rudimentary driver for such chips (atmel-ecc.c for ATECC508A), but it doesn't implement the sign operation. The kernel would first have to grow support for a hardware asymmetric_key_subtype and at least one driver implementing the sign operation before exposure to guests via virtio makes sense. Signed-off-by: Lukas Wunner <lukas@wunner.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-07-17virtio: rename virtio_find_vqs_info() to virtio_find_vqs()Jiri Pirko-1/+1
Since the original virtio_find_vqs() is no longer present, rename virtio_find_vqs_info() back to virtio_find_vqs(). Signed-off-by: Jiri Pirko <jiri@nvidia.com> Message-Id: <20240708074814.1739223-20-jiri@resnulli.us> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2024-07-17virtio_crypto: convert to use virtio_find_vqs_info()Jiri Pirko-19/+12
Instead of passing separate names and callbacks arrays to virtio_find_vqs(), allocate one of virtual_queue_info structs and pass it to virtio_find_vqs_info(). Suggested-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com> Signed-off-by: Jiri Pirko <jiri@nvidia.com> Message-Id: <20240708074814.1739223-13-jiri@resnulli.us> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2024-05-22crypto: virtio - drop owner assignmentKrzysztof Kozlowski-1/+0
virtio core already sets the .owner, so driver does not need to. Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Message-Id: <20240331-module-owner-virtio-v2-10-98f04bfaf46a@linaro.org> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-03-15Merge tag 'v6.9-p1' of ↵Linus Torvalds-8/+6
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto updates from Herbert Xu: "API: - Avoid unnecessary copying in scomp for trivial SG lists Algorithms: - Optimise NEON CCM implementation on ARM64 Drivers: - Add queue stop/query debugfs support in hisilicon/qm - Intel qat updates and cleanups" * tag 'v6.9-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (79 commits) Revert "crypto: remove CONFIG_CRYPTO_STATS" crypto: scomp - remove memcpy if sg_nents is 1 and pages are lowmem crypto: tcrypt - add ffdhe2048(dh) test crypto: iaa - fix the missing CRYPTO_ALG_ASYNC in cra_flags crypto: hisilicon/zip - fix the missing CRYPTO_ALG_ASYNC in cra_flags hwrng: hisi - use dev_err_probe MAINTAINERS: Remove T Ambarus from few mchp entries crypto: iaa - Fix comp/decomp delay statistics crypto: iaa - Fix async_disable descriptor leak dt-bindings: rng: atmel,at91-trng: add sam9x7 TRNG dt-bindings: crypto: add sam9x7 in Atmel TDES dt-bindings: crypto: add sam9x7 in Atmel SHA dt-bindings: crypto: add sam9x7 in Atmel AES crypto: remove CONFIG_CRYPTO_STATS crypto: dh - Make public key test FIPS-only crypto: rockchip - fix to check return value crypto: jitter - fix CRYPTO_JITTERENTROPY help text crypto: qat - make ring to service map common for QAT GEN4 crypto: qat - fix ring to service map for dcc in 420xx crypto: qat - fix ring to service map for dcc in 4xxx ...
2024-02-09crypto: virtio - remove duplicate check if queue is brokenLi RongQing-2/+0
virtqueue_enable_cb() will call virtqueue_poll() which will check if queue is broken at beginning, so remove the virtqueue_is_broken() call Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Li RongQing <lirongqing@baidu.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09crypto: virtio/akcipher - Fix stack overflow on memcpyzhenwei pi-2/+3
sizeof(struct virtio_crypto_akcipher_session_para) is less than sizeof(struct virtio_crypto_op_ctrl_req::u), copying more bytes from stack variable leads stack overflow. Clang reports this issue by commands: make -j CC=clang-14 mrproper >/dev/null 2>&1 make -j O=/tmp/crypto-build CC=clang-14 allmodconfig >/dev/null 2>&1 make -j O=/tmp/crypto-build W=1 CC=clang-14 drivers/crypto/virtio/ virtio_crypto_akcipher_algs.o Fixes: 59ca6c93387d ("virtio-crypto: implement RSA algorithm") Link: https://lore.kernel.org/all/0a194a79-e3a3-45e7-be98-83abd3e1cb7e@roeck-us.net/ Cc: <stable@vger.kernel.org> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Tested-by: Nathan Chancellor <nathan@kernel.org> # build Acked-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-01-26crypto: virtio - Less function calls in __virtio_crypto_akcipher_do_req() ↵Markus Elfring-6/+6
after error detection The kfree() function was called in up to two cases by the __virtio_crypto_akcipher_do_req() function during error handling even if the passed variable contained a null pointer. This issue was detected by using the Coccinelle software. * Adjust jump targets. * Delete two initialisations which became unnecessary with this refactoring. Signed-off-by: Markus Elfring <elfring@users.sourceforge.net> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Justin Stitt <justinstitt@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-12-22crypto: virtio - Wait for tasklet to complete on device removewangyangxin-0/+3
The scheduled tasklet needs to be executed on device remove. Fixes: fed93fb62e05 ("crypto: virtio - Handle dataq logic with tasklet") Signed-off-by: wangyangxin <wangyangxin1@huawei.com> Signed-off-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-12-01crypto: virtio - Handle dataq logic with taskletGonglei (Arei)-10/+15
Doing ipsec produces a spinlock recursion warning. This is due to crypto_finalize_request() being called in the upper half. Move virtual data queue processing of virtio-crypto driver to tasklet. Fixes: dbaf0624ffa57 ("crypto: add virtio-crypto driver") Reported-by: Halil Pasic <pasic@linux.ibm.com> Signed-off-by: wangyangxin <wangyangxin1@huawei.com> Signed-off-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-18virtio-crypto: handle config changed by work queuezhenwei pi-1/+16
MST pointed out: config change callback is also handled incorrectly in this driver, it takes a mutex from interrupt context. Handle config changed by work queue instead. Cc: stable@vger.kernel.org Cc: Gonglei (Arei) <arei.gonglei@huawei.com> Cc: Halil Pasic <pasic@linux.ibm.com> Cc: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20231007064309.844889-1-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-08-18crypto: virtio - Use new crypto_engine_op interfaceHerbert Xu-26/+30
Use the new crypto_engine_op interface where the callback is stored in the algorithm object. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-08-18crypto: virtio - Remove prepare/unprepare requestHerbert Xu-4/+0
The callbacks for prepare and unprepare request in crypto_engine is superfluous. They can be done directly from do_one_request. Move the code into do_one_request and remove the unused callbacks. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-07-03virtio-crypto: call scheduler when we free unused buffsXianting Tian-0/+1
For virtio-net we were getting CPU stall warnings, and fixed it by calling the scheduler: see f8bb51043945 ("virtio_net: suppress cpu stall when free_unused_bufs"). This driver is similar so theoretically the same logic applies. Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com> Message-Id: <20230609131817.712867-2-xianting.tian@linux.alibaba.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-02-21Merge tag 'v6.3-p1' of ↵Linus Torvalds-1/+1
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 Pull crypto update from Herbert Xu: "API: - Use kmap_local instead of kmap_atomic - Change request callback to take void pointer - Print FIPS status in /proc/crypto (when enabled) Algorithms: - Add rfc4106/gcm support on arm64 - Add ARIA AVX2/512 support on x86 Drivers: - Add TRNG driver for StarFive SoC - Delete ux500/hash driver (subsumed by stm32/hash) - Add zlib support in qat - Add RSA support in aspeed" * tag 'v6.3-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (156 commits) crypto: x86/aria-avx - Do not use avx2 instructions crypto: aspeed - Fix modular aspeed-acry crypto: hisilicon/qm - fix coding style issues crypto: hisilicon/qm - update comments to match function crypto: hisilicon/qm - change function names crypto: hisilicon/qm - use min() instead of min_t() crypto: hisilicon/qm - remove some unused defines crypto: proc - Print fips status crypto: crypto4xx - Call dma_unmap_page when done crypto: octeontx2 - Fix objects shared between several modules crypto: nx - Fix sparse warnings crypto: ecc - Silence sparse warning tls: Pass rec instead of aead_req into tls_encrypt_done crypto: api - Remove completion function scaffolding tls: Remove completion function scaffolding tipc: Remove completion function scaffolding net: ipv6: Remove completion function scaffolding net: ipv4: Remove completion function scaffolding net: macsec: Remove completion function scaffolding dm: Remove completion function scaffolding ...
2023-02-10crypto: virtio/akcipher - Do not use GFP_ATOMIC when not neededChristophe JAILLET-1/+1
There is no need to use GFP_ATOMIC here. GFP_KERNEL is already used for another memory allocation just the line after. Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-12-28virtio-crypto: fix memory leak in virtio_crypto_alg_skcipher_close_session()Wei Yongjun-1/+2
'vc_ctrl_req' is alloced in virtio_crypto_alg_skcipher_close_session(), and should be freed in the invalid ctrl_status->status error handling case. Otherwise there is a memory leak. Fixes: 0756ad15b1fe ("virtio-crypto: use private buffer for control request") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Message-Id: <20221114110740.537276-1-weiyongjun@huaweicloud.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Acked-by: zhenwei pi<pizhenwei@bytedance.com> Acked-by: Jason Wang <jasowang@redhat.com>
2022-12-02crypto: virtio - Use helper to set reqsizeHerbert Xu-2/+3
The value of reqsize must only be changed through the helper. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Acked-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-09-27virtio-crypto: fix memory-leaklei he-0/+4
Fix memory-leak for virtio-crypto akcipher request, this problem is introduced by 59ca6c93387d3(virtio-crypto: implement RSA algorithm). The leak can be reproduced and tested with the following script inside virtual machine: #!/bin/bash LOOP_TIMES=10000 # required module: pkcs8_key_parser, virtio_crypto modprobe pkcs8_key_parser # if CONFIG_PKCS8_PRIVATE_KEY_PARSER=m modprobe virtio_crypto # if CONFIG_CRYPTO_DEV_VIRTIO=m rm -rf /tmp/data dd if=/dev/random of=/tmp/data count=1 bs=230 # generate private key and self-signed cert openssl req -nodes -x509 -newkey rsa:2048 -keyout key.pem \ -outform der -out cert.der \ -subj "/C=CN/ST=GD/L=SZ/O=vihoo/OU=dev/CN=always.com/emailAddress=yy@always.com" # convert private key from pem to der openssl pkcs8 -in key.pem -topk8 -nocrypt -outform DER -out key.der # add key PRIV_KEY_ID=`cat key.der | keyctl padd asymmetric test_priv_key @s` echo "priv key id = "$PRIV_KEY_ID PUB_KEY_ID=`cat cert.der | keyctl padd asymmetric test_pub_key @s` echo "pub key id = "$PUB_KEY_ID # query key keyctl pkey_query $PRIV_KEY_ID 0 keyctl pkey_query $PUB_KEY_ID 0 # here we only run pkey_encrypt becasuse it is the fastest interface function bench_pub() { keyctl pkey_encrypt $PUB_KEY_ID 0 /tmp/data enc=pkcs1 >/tmp/enc.pub } # do bench_pub in loop to obtain the memory leak for (( i = 0; i < ${LOOP_TIMES}; ++i )); do bench_pub done Signed-off-by: lei he <helei.sig11@bytedance.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Message-Id: <20220919075158.3625-1-helei.sig11@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-31virtio-crypto: enable retry for virtio-crypto-devlei he-1/+2
Enable retry for virtio-crypto-dev, so that crypto-engine can process cipher-requests parallelly. Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: lei he <helei.sig11@bytedance.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20220506131627.180784-6-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-31virtio-crypto: adjust dst_len at ops callbacklei he-1/+4
For some akcipher operations(eg, decryption of pkcs1pad(rsa)), the length of returned result maybe less than akcipher_req->dst_len, we need to recalculate the actual dst_len through the virt-queue protocol. Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: lei he <helei.sig11@bytedance.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20220506131627.180784-5-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-31virtio-crypto: wait ctrl queue instead of busy pollingzhenwei pi-55/+64
Originally, after submitting request into virtio crypto control queue, the guest side polls the result from the virt queue. This works like following: CPU0 CPU1 ... CPUx CPUy | | | | \ \ / / \--------spin_lock(&vcrypto->ctrl_lock)-------/ | virtqueue add & kick | busy poll virtqueue | spin_unlock(&vcrypto->ctrl_lock) ... There are two problems: 1, The queue depth is always 1, the performance of a virtio crypto device gets limited. Multi user processes share a single control queue, and hit spin lock race from control queue. Test on Intel Platinum 8260, a single worker gets ~35K/s create/close session operations, and 8 workers get ~40K/s operations with 800% CPU utilization. 2, The control request is supposed to get handled immediately, but in the current implementation of QEMU(v6.2), the vCPU thread kicks another thread to do this work, the latency also gets unstable. Tracking latency of virtio_crypto_alg_akcipher_close_session in 5s: usecs : count distribution 0 -> 1 : 0 | | 2 -> 3 : 7 | | 4 -> 7 : 72 | | 8 -> 15 : 186485 |************************| 16 -> 31 : 687 | | 32 -> 63 : 5 | | 64 -> 127 : 3 | | 128 -> 255 : 1 | | 256 -> 511 : 0 | | 512 -> 1023 : 0 | | 1024 -> 2047 : 0 | | 2048 -> 4095 : 0 | | 4096 -> 8191 : 0 | | 8192 -> 16383 : 2 | | This means that a CPU may hold vcrypto->ctrl_lock as long as 8192~16383us. To improve the performance of control queue, a request on control queue waits completion instead of busy polling to reduce lock racing, and gets completed by control queue callback. CPU0 CPU1 ... CPUx CPUy | | | | \ \ / / \--------spin_lock(&vcrypto->ctrl_lock)-------/ | virtqueue add & kick | ---------spin_unlock(&vcrypto->ctrl_lock)------ / / \ \ | | | | wait wait wait wait Test this patch, the guest side get ~200K/s operations with 300% CPU utilization. Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20220506131627.180784-4-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-31virtio-crypto: use private buffer for control requestzhenwei pi-45/+79
Originally, all of the control requests share a single buffer( ctrl & input & ctrl_status fields in struct virtio_crypto), this allows queue depth 1 only, the performance of control queue gets limited by this design. In this patch, each request allocates request buffer dynamically, and free buffer after request, so the scope protected by ctrl_lock also get optimized here. It's possible to optimize control queue depth in the next step. A necessary comment is already in code, still describe it again: /* * Note: there are padding fields in request, clear them to zero before * sending to host to avoid to divulge any information. * Ex, virtio_crypto_ctrl_request::ctrl::u::destroy_session::padding[48] */ So use kzalloc to allocate buffer of struct virtio_crypto_ctrl_request. Potentially dereferencing uninitialized variables: Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20220506131627.180784-3-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-31virtio-crypto: change code stylezhenwei pi-53/+59
Use temporary variable to make code easy to read and maintain. /* Pad cipher's parameters */ vcrypto->ctrl.u.sym_create_session.op_type = cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo = vcrypto->ctrl.header.algo; vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen = cpu_to_le32(keylen); vcrypto->ctrl.u.sym_create_session.u.cipher.para.op = cpu_to_le32(op); --> sym_create_session = &ctrl->u.sym_create_session; sym_create_session->op_type = cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); sym_create_session->u.cipher.para.algo = ctrl->header.algo; sym_create_session->u.cipher.para.keylen = cpu_to_le32(keylen); sym_create_session->u.cipher.para.op = cpu_to_le32(op); The new style shows more obviously: - the variable we want to operate. - an assignment statement in a single line. Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Gonglei <arei.gonglei@huawei.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Message-Id: <20220506131627.180784-2-pizhenwei@bytedance.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-03-28virtio-crypto: rename skcipher algszhenwei pi-9/+9
Suggested by Gonglei, rename virtio_crypto_algs.c to virtio_crypto_skcipher_algs.c. Also minor changes for function name. Thus the function of source files get clear: skcipher services in virtio_crypto_skcipher_algs.c and akcipher services in virtio_crypto_akcipher_algs.c. Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Link: https://lore.kernel.org/r/20220302033917.1295334-5-pizhenwei@bytedance.com Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Gonglei <arei.gonglei@huawei.com>
2022-03-28virtio-crypto: implement RSA algorithmzhenwei pi-1/+608
Support rsa & pkcs1pad(rsa,sha1) with priority 150. Test with QEMU built-in backend, it works fine. 1, The self-test framework of crypto layer works fine in guest kernel 2, Test with Linux guest(with asym support), the following script test(note that pkey_XXX is supported only in a newer version of keyutils): - both public key & private key - create/close session - encrypt/decrypt/sign/verify basic driver operation - also test with kernel crypto layer(pkey add/query) All the cases work fine. rm -rf *.der *.pem *.pfx modprobe pkcs8_key_parser # if CONFIG_PKCS8_PRIVATE_KEY_PARSER=m rm -rf /tmp/data dd if=/dev/random of=/tmp/data count=1 bs=226 openssl req -nodes -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -subj "/C=CN/ST=BJ/L=HD/O=qemu/OU=dev/CN=qemu/emailAddress=qemu@qemu.org" openssl pkcs8 -in key.pem -topk8 -nocrypt -outform DER -out key.der openssl x509 -in cert.pem -inform PEM -outform DER -out cert.der PRIV_KEY_ID=`cat key.der | keyctl padd asymmetric test_priv_key @s` echo "priv key id = "$PRIV_KEY_ID PUB_KEY_ID=`cat cert.der | keyctl padd asymmetric test_pub_key @s` echo "pub key id = "$PUB_KEY_ID keyctl pkey_query $PRIV_KEY_ID 0 keyctl pkey_query $PUB_KEY_ID 0 echo "Enc with priv key..." keyctl pkey_encrypt $PRIV_KEY_ID 0 /tmp/data enc=pkcs1 >/tmp/enc.priv echo "Dec with pub key..." keyctl pkey_decrypt $PRIV_KEY_ID 0 /tmp/enc.priv enc=pkcs1 >/tmp/dec cmp /tmp/data /tmp/dec echo "Sign with priv key..." keyctl pkey_sign $PRIV_KEY_ID 0 /tmp/data enc=pkcs1 hash=sha1 > /tmp/sig echo "Verify with pub key..." keyctl pkey_verify $PRIV_KEY_ID 0 /tmp/data /tmp/sig enc=pkcs1 hash=sha1 echo "Enc with pub key..." keyctl pkey_encrypt $PUB_KEY_ID 0 /tmp/data enc=pkcs1 >/tmp/enc.pub echo "Dec with priv key..." keyctl pkey_decrypt $PRIV_KEY_ID 0 /tmp/enc.pub enc=pkcs1 >/tmp/dec cmp /tmp/data /tmp/dec echo "Verify with pub key..." keyctl pkey_verify $PUB_KEY_ID 0 /tmp/data /tmp/sig enc=pkcs1 hash=sha1 [1 compiling warning during development] Reported-by: kernel test robot <lkp@intel.com> Co-developed-by: lei he <helei.sig11@bytedance.com> Signed-off-by: lei he <helei.sig11@bytedance.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> Link: https://lore.kernel.org/r/20220302033917.1295334-4-pizhenwei@bytedance.com Reviewed-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Nathan Chancellor <nathan@kernel.org> #Kconfig tweaks Link: https://lore.kernel.org/r/20220308205309.2192502-1-nathan@kernel.org Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-01-14virtio: wrap config->reset callsMichael S. Tsirkin-4/+4
This will enable cleanups down the road. The idea is to disable cbs, then add "flush_queued_cbs" callback as a parameter, this way drivers can flush any work queued after callbacks have been disabled. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Link: https://lore.kernel.org/r/20211013105226.20225-1-mst@redhat.com Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2021-08-12crypto: virtio - Replace deprecated CPU-hotplug functions.Sebastian Andrzej Siewior-2/+2
The functions get_online_cpus() and put_online_cpus() have been deprecated during the CPU hotplug rework. They map directly to cpus_read_lock() and cpus_read_unlock(). Replace deprecated CPU-hotplug functions with the official version. The behavior remains unchanged. Cc: Gonglei <arei.gonglei@huawei.com> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: virtualization@lists.linux-foundation.org Cc: linux-crypto@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-08-21crypto: virtio - don't use 'default m'Ram Muthiah-1/+0
Drivers shouldn't be enabled by default unless there is a very good reason to do so. There doesn't seem to be any such reason for the virtio crypto driver, so change it to the default of 'n'. Signed-off-by: Ram Muthiah <rammuthiah@google.com> [EB: adjusted commit message] Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-08-11Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhostLinus Torvalds-23/+23
Pull virtio updates from Michael Tsirkin: - IRQ bypass support for vdpa and IFC - MLX5 vdpa driver - Endianness fixes for virtio drivers - Misc other fixes * tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost: (71 commits) vdpa/mlx5: fix up endian-ness for mtu vdpa: Fix pointer math bug in vdpasim_get_config() vdpa/mlx5: Fix pointer math in mlx5_vdpa_get_config() vdpa/mlx5: fix memory allocation failure checks vdpa/mlx5: Fix uninitialised variable in core/mr.c vdpa_sim: init iommu lock virtio_config: fix up warnings on parisc vdpa/mlx5: Add VDPA driver for supported mlx5 devices vdpa/mlx5: Add shared memory registration code vdpa/mlx5: Add support library for mlx5 VDPA implementation vdpa/mlx5: Add hardware descriptive header file vdpa: Modify get_vq_state() to return error code net/vdpa: Use struct for set/get vq state vdpa: remove hard coded virtq num vdpasim: support batch updating vhost-vdpa: support IOTLB batching hints vhost-vdpa: support get/set backend features vhost: generialize backend features setting/getting vhost-vdpa: refine ioctl pre-processing vDPA: dont change vq irq after DRIVER_OK ...
2020-08-07mm, treewide: rename kzfree() to kfree_sensitive()Waiman Long-7/+7
As said by Linus: A symmetric naming is only helpful if it implies symmetries in use. Otherwise it's actively misleading. In "kzalloc()", the z is meaningful and an important part of what the caller wants. In "kzfree()", the z is actively detrimental, because maybe in the future we really _might_ want to use that "memfill(0xdeadbeef)" or something. The "zero" part of the interface isn't even _relevant_. The main reason that kzfree() exists is to clear sensitive information that should not be leaked to other future users of the same memory objects. Rename kzfree() to kfree_sensitive() to follow the example of the recently added kvfree_sensitive() and make the intention of the API more explicit. In addition, memzero_explicit() is used to clear the memory to make sure that it won't get optimized away by the compiler. The renaming is done by using the command sequence: git grep -w --name-only kzfree |\ xargs sed -i 's/kzfree/kfree_sensitive/' followed by some editing of the kfree_sensitive() kerneldoc and adding a kzfree backward compatibility macro in slab.h. [akpm@linux-foundation.org: fs/crypto/inline_crypt.c needs linux/slab.h] [akpm@linux-foundation.org: fix fs/crypto/inline_crypt.c some more] Suggested-by: Joe Perches <joe@perches.com> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Howells <dhowells@redhat.com> Acked-by: Michal Hocko <mhocko@suse.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> Cc: James Morris <jmorris@namei.org> Cc: "Serge E. Hallyn" <serge@hallyn.com> Cc: Joe Perches <joe@perches.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: David Rientjes <rientjes@google.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Cc: "Jason A . Donenfeld" <Jason@zx2c4.com> Link: http://lkml.kernel.org/r/20200616154311.12314-3-longman@redhat.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-05virtio_crypto: convert to LE accessorsMichael S. Tsirkin-23/+23
Virtio crypto is modern-only. Use LE accessors for config space. Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-07-16crypto: drivers - set the flag CRYPTO_ALG_ALLOCATES_MEMORYMikulas Patocka-1/+2
Set the flag CRYPTO_ALG_ALLOCATES_MEMORY in the crypto drivers that allocate memory. drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c: sun8i_ce_cipher drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c: sun8i_ss_cipher drivers/crypto/amlogic/amlogic-gxl-core.c: meson_cipher drivers/crypto/axis/artpec6_crypto.c: artpec6_crypto_common_init drivers/crypto/bcm/cipher.c: spu_skcipher_rx_sg_create drivers/crypto/caam/caamalg.c: aead_edesc_alloc drivers/crypto/caam/caamalg_qi.c: aead_edesc_alloc drivers/crypto/caam/caamalg_qi2.c: aead_edesc_alloc drivers/crypto/caam/caamhash.c: hash_digest_key drivers/crypto/cavium/cpt/cptvf_algs.c: process_request drivers/crypto/cavium/nitrox/nitrox_aead.c: nitrox_process_se_request drivers/crypto/cavium/nitrox/nitrox_skcipher.c: nitrox_process_se_request drivers/crypto/ccp/ccp-crypto-aes-cmac.c: ccp_do_cmac_update drivers/crypto/ccp/ccp-crypto-aes-galois.c: ccp_crypto_enqueue_request drivers/crypto/ccp/ccp-crypto-aes-xts.c: ccp_crypto_enqueue_request drivers/crypto/ccp/ccp-crypto-aes.c: ccp_crypto_enqueue_request drivers/crypto/ccp/ccp-crypto-des3.c: ccp_crypto_enqueue_request drivers/crypto/ccp/ccp-crypto-sha.c: ccp_crypto_enqueue_request drivers/crypto/chelsio/chcr_algo.c: create_cipher_wr drivers/crypto/hisilicon/sec/sec_algs.c: sec_alloc_and_fill_hw_sgl drivers/crypto/hisilicon/sec2/sec_crypto.c: sec_alloc_req_id drivers/crypto/inside-secure/safexcel_cipher.c: safexcel_queue_req drivers/crypto/inside-secure/safexcel_hash.c: safexcel_ahash_enqueue drivers/crypto/ixp4xx_crypto.c: ablk_perform drivers/crypto/marvell/cesa/cipher.c: mv_cesa_skcipher_dma_req_init drivers/crypto/marvell/cesa/hash.c: mv_cesa_ahash_dma_req_init drivers/crypto/marvell/octeontx/otx_cptvf_algs.c: create_ctx_hdr drivers/crypto/n2_core.c: n2_compute_chunks drivers/crypto/picoxcell_crypto.c: spacc_sg_to_ddt drivers/crypto/qat/qat_common/qat_algs.c: qat_alg_skcipher_encrypt drivers/crypto/qce/skcipher.c: qce_skcipher_async_req_handle drivers/crypto/talitos.c : talitos_edesc_alloc drivers/crypto/virtio/virtio_crypto_algs.c: __virtio_crypto_skcipher_do_req drivers/crypto/xilinx/zynqmp-aes-gcm.c: zynqmp_aes_aead_cipher Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> [EB: avoid overly-long lines] Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09crypto: virtio - constify features[] and id_table[]Rikard Falkeborn-2/+2
features[] and id_table[] are not modified and can be made const to allow the compiler to put them in read-only memory. Before: text data bss dec hex filename 11534 2056 160 13750 35b6 drivers/crypto/virtio/virtio_crypto_core.o After: text data bss dec hex filename 11630 1992 128 13750 35b6 drivers/crypto/virtio/virtio_crypto_core.o Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-06-04crypto: virtio: Fix dest length calculation in __virtio_crypto_skcipher_do_req()Longpeng(Mike)-0/+1
The src/dst length is not aligned with AES_BLOCK_SIZE(which is 16) in some testcases in tcrypto.ko. For example, the src/dst length of one of cts(cbc(aes))'s testcase is 17, the crypto_virtio driver will set @src_data_len=16 but @dst_data_len=17 in this case and get a wrong at then end. SRC: pp pp pp pp pp pp pp pp pp pp pp pp pp pp pp pp pp (17 bytes) EXP: cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc pp (17 bytes) DST: cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc 00 (pollute the last bytes) (pp: plaintext cc:ciphertext) Fix this issue by limit the length of dest buffer. Fixes: dbaf0624ffa5 ("crypto: add virtio-crypto driver") Cc: Gonglei <arei.gonglei@huawei.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: virtualization@lists.linux-foundation.org Cc: linux-kernel@vger.kernel.org Cc: stable@vger.kernel.org Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com> Link: https://lore.kernel.org/r/20200602070501.2023-4-longpeng2@huawei.com Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-06-04crypto: virtio: Fix use-after-free in virtio_crypto_skcipher_finalize_req()Longpeng(Mike)-2/+3
The system'll crash when the users insmod crypto/tcrypto.ko with mode=155 ( testing "authenc(hmac(sha1),cbc(aes))" ). It's caused by reuse the memory of request structure. In crypto_authenc_init_tfm(), the reqsize is set to: [PART 1] sizeof(authenc_request_ctx) + [PART 2] ictx->reqoff + [PART 3] MAX(ahash part, skcipher part) and the 'PART 3' is used by both ahash and skcipher in turn. When the virtio_crypto driver finish skcipher req, it'll call ->complete callback(in crypto_finalize_skcipher_request) and then free its resources whose pointers are recorded in 'skcipher parts'. However, the ->complete is 'crypto_authenc_encrypt_done' in this case, it will use the 'ahash part' of the request and change its content, so virtio_crypto driver will get the wrong pointer after ->complete finish and mistakenly free some other's memory. So the system will crash when these memory will be used again. The resources which need to be cleaned up are not used any more. But the pointers of these resources may be changed in the function "crypto_finalize_skcipher_request". Thus release specific resources before calling this function. Fixes: dbaf0624ffa5 ("crypto: add virtio-crypto driver") Reported-by: LABBE Corentin <clabbe@baylibre.com> Cc: Gonglei <arei.gonglei@huawei.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: virtualization@lists.linux-foundation.org Cc: linux-kernel@vger.kernel.org Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20200123101000.GB24255@Red Acked-by: Gonglei <arei.gonglei@huawei.com> Signed-off-by: Longpeng(Mike) <longpeng2@huawei.com> Link: https://lore.kernel.org/r/20200602070501.2023-3-longpeng2@huawei.com Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
generated by cgit v1.2.3 (git 2.46.0) at 2026-03-19 15:01:22 +0000