[Feature Request] Update INSTALL.md for AMD ROCm for Linux builds #5522
Comments
|
Thank you for reporting this!
We don't currently have an I'm not up to date on this, but I guess not everyone with an AMD card will use their ROCm driver now - they still also have their "Pro" driver, correct? So an instructions update should cover both cases. Can you suggest a specific edit? |
|
Noted. I'll work on it and provide a detailed list of edits required. I think it'll be easier to work on a PR hence will do that. |
|
Looks like you never said what version(s) of John you observed the problem and tested the fix with?
In other words, we're now (almost) able to build with OpenCL support without actually finding any OpenCL at build time. The only reasons we don't do this by default are to save some build time and avoid the risk of some build issues on systems where OpenCL support wouldn't be needed. |
|
Also, if we find any OpenCL library at build time, then the resulting build should work with whatever (maybe other) OpenCL library it sees at run time. So rather than suggest e.g. ROCm-specific library path, we could suggest installing generic OpenCL library from the distro's packages. In fact, we already make this sort of suggestion in Did this not work for you? What operating system are you on? |
|
Apologies. I completely overlooked those UBUNTU-specific instructions. I did try them now and it worked as expected. However, for Vega GPUs and later, AMD recommends using ROCm installation for compute applications. They provide the GPU driver in their newer method of GPU driver installation, which is using This includes both, HIP and OpenCL. It seems a little redundant to install OpenCL headers separately when they are included with ROCm installation itself. |
The question is which driver actually performs better with our kernels. While we're able to get 100% working with NVIDIA drivers (with occasional exceptions), for AMD there are always some failing (usually miscompiles, resulting in failed self-tests). So if one AMD driver has e.g. 5 JtR formats failing and another 15, we'd "recommend" the one with 5. You could want to run
OpenCL headers installed on the system (in whatever way) aren't required/used by current code in this repo at all! This is stale documentation, we may want to update it to drop that. OTOH, I suspect that some people building the older 1.9.0-jumbo-1 release also end up looking at the documentation in here, and for them such update would prevent the builds from working. |
|
Interesting findings on further testing:
john_opencl_perf_benchmark.txt Looking into them might be another investigation altogether.
|
|
Thank you very much for running these tests.
Can you show this (some copy-paste from the terminal from just before the segfault)?
Can you provide more detail on these? We normally log the function name/parameter where the failure was observed.
This and the same set of failing formats suggests these drivers are probably the same under the hood now.
We shouldn't need |
|
In case of AMD, the system [1] uses some environment variables [2] to detect where things are installed. It seems that your system uses new names or something [3]. [1] the OpenCL automatic "configure" detection system. |
|
@solardiz here is the output from failed algorithms: I'll provide a strace output soon as well. no new env variables were present. |
From 60eb23226ef2796207eed15c86ff12d8bc8ee992 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Claudio=20Andr=C3=A9?= <[email protected]>
Date: Wed, 14 Aug 2024 13:20:45 -0300
Subject: [PATCH] OpenCL: add ROCm installation detection
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The OpenCL detection mechanisms used by `configure` do not work properly
for the ROCm driver, this patch probably fixes the problem (untested).
Signed-off-by: Claudio André <[email protected]>
---
src/m4/jtr_utility_macros.m4 | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/m4/jtr_utility_macros.m4 b/src/m4/jtr_utility_macros.m4
index 6b5c58e2b..edea65215 100644
--- a/src/m4/jtr_utility_macros.m4
+++ b/src/m4/jtr_utility_macros.m4
@@ -213,6 +213,14 @@ AC_DEFUN([JTR_SET_OPENCL_INCLUDES],
AC_MSG_CHECKING([additional paths for OpenCL])
ADD_LDFLAGS=""
ADD_CFLAGS=""
+ ROCM_INSTALLED="/opt/rocm"
+ if test -d "$ROCM_INSTALLED"; then
+ if test $CPU_BIT_STR = 64 -a -d "$ROCM_INSTALLED/lib/x86_64" ; then
+ ADD_LDFLAGS="$ADD_LDFLAGS -L$ROCM_INSTALLED/lib/x86_64"
+ elif test -d "$ROCM_INSTALLED/lib"; then
+ ADD_LDFLAGS="$ADD_LDFLAGS -L$ROCM_INSTALLED/lib"
+ fi
+ fi
if test -n "$AMDAPPSDKROOT"; then
if test -d "$AMDAPPSDKROOT/include"; then
ADD_CFLAGS="$ADD_CFLAGS -I$AMDAPPSDKROOT/include"
--
2.43.0
|
|
Thank you @claudioandre-br!
I'm pretty sure @mostwanted002 is on 64-bit, yet didn't need to specify the |
With the latest ROCm installation, AMD has restructured OpenCL headers and libraries.
Just specifying 2 flags in
./configurecommand can fix that.It'll be great if we can add a small section in INSTALL.md for AMD GPU build:
With default
./configure:# john-the-ripper.opencl --list=opencl-devices Error: No OpenCL-capable platforms were detected by the installed OpenCL driver. Error: No OpenCL-capable devices were detected by the installed OpenCL driver.With the updated configure:
The text was updated successfully, but these errors were encountered: