Operation Error: /usr/bin/ld: cannot find -lcuda

[ying123ww]

Linker Cannot Find -lcuda When Using Conda Environment's CUDA

Description

I'm encountering an error when using flash-attn in my Conda environment. Despite having CUDA PyTorch and flash-attention properly installed within the environment and setting the appropriate environment variables, the linker cannot find -lcuda, resulting in a failed operation.

Installation Steps

1. Install CUDA in Conda Environment

   conda install nvidia/label/cuda-12.1.0::cuda-toolkit

2. Install PyTorch with CUDA Support

   pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu121

3. Install FlashAttention

   pip install flash-attn --no-build-isolation --index-url https://pypi.org/simple --no-cache-dir

4. Resolve Missing Dependencies

   pip install psutil

5. Re-install FlashAttention

   pip install flash-attn --no-build-isolation --index-url https://pypi.org/simple --no-cache-dir

6. Install Additional Requirements

   pip install -r requirements.txt

Environment Variables Set

I have set the following environment variables to point to the CUDA installation in my Conda environment:

export CUDA_HOME=/home/yxy/miniconda3/envs/sedd_new  # Path to my Conda environment
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH

Error Message
I want the program to use libcuda.so in my conda environment instead of my system
During operation, I receive the following error:

/usr/bin/ld: skipping incompatible /lib/i386-linux-gnu/libcuda.so when searching for -lcuda
/usr/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status

Unfortunately, it's still looking for lcuda in my system

Investigation

• The linker /usr/bin/ld is being used, which links to the 64-bit version:

  /usr/bin/ld: symbolic link to x86_64-linux-gnu-ld
  /usr/bin/x86_64-linux-gnu-ld: symbolic link to x86_64-linux-gnu-ld.bfd
  /usr/bin/x86_64-linux-gnu-ld.bfd: ELF 64-bit LSB shared object

• Locations of libcuda.so on my system:

  • System libcuda.so files:

    /usr/local/cuda-12.5/targets/x86_64-linux/lib/stubs/libcuda.so
    /usr/lib/i386-linux-gnu/libcuda.so

  • Conda environment libcuda.so file:

    /home/yxy/miniconda3/envs/sedd_new/lib/stubs/libcuda.so

• Verified the nvcc version (desired CUDA version in Conda environment):

  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2023 NVIDIA Corporation
  Built on Tue_Feb__7_19:32:13_PST_2023
  Cuda compilation tools, release 12.1, V12.1.66
  Build cuda_12.1.r12.1/compiler.32415258_0

Issue

It appears that when I run the program ('flash-attn' is installed successfully), the linker is not using the libcuda.so from my Conda environment but is instead searching the system directories and failing to find a compatible version. I want flash-attn to use the libcuda.so located in my Conda environment at /home/yxy/miniconda3/envs/sedd_new/lib/stubs/libcuda.so.

Request

Could you please help me resolve this issue? How can I configure the installation process so that flash-attn uses the libcuda.so from my Conda environment instead of the system's version?

Thank you for your assistance!