RUST笔记：candle使用基础_candle rust

candle介绍

• candle是huggingface开源的Rust的极简 ML 框架。

candle-矩阵乘法示例

cargo new myapp
cd myapp
cargo add --git https://github.com/huggingface/candle.git candle-core
cargo build # 测试，或执行 cargo ckeck
1
2
3
4

• main.rs

use candle_core::{Device, Tensor};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let device = Device::Cpu;

    let a = Tensor::randn(0f32, 1., (2, 3), &device)?;
    let b = Tensor::randn(0f32, 1., (3, 4), &device)?;

    let c = a.matmul(&b)?;
    println!("{c}");
    Ok(())
}

1
2
3
4
5
6
7
8
9
10
11
12
13

• 项目输出

~/myrust$ cargo new myapp
     Created binary (application) `myapp` package
~/myrust$ cd myapp
~/myrust/myapp$ cargo add --git https://github.com/huggingface/candle.git candle-core
    Updating git repository `https://github.com/huggingface/candle.git`
    Updating git submodule `https://github.com/NVIDIA/cutlass.git`
      Adding candle-core (git) to dependencies.
             Features:
             - accelerate
             - cuda
             - cudarc
             - cudnn
             - metal
             - mkl
    Updating git repository `https://github.com/huggingface/candle.git`
    Updating crates.io index
~/myrust/myapp$ cargo build
  Downloaded serde_derive v1.0.195
  Downloaded either v1.9.0
  Downloaded autocfg v1.1.0
  Downloaded zerofrom v0.1.3
  Downloaded zerofrom-derive v0.1.3
  Downloaded synstructure v0.13.0
  Downloaded crossbeam-deque v0.8.5
  Downloaded yoke-derive v0.7.3
  Downloaded half v2.3.1
  Downloaded bytemuck v1.14.1
  Downloaded rand_core v0.6.4
  Downloaded paste v1.0.14
  Downloaded proc-macro2 v1.0.78
  Downloaded itoa v1.0.10
  Downloaded memmap2 v0.9.4
  Downloaded syn v2.0.48
  Downloaded crossbeam-epoch v0.9.18
  Downloaded cfg-if v1.0.0
  Downloaded bitflags v1.3.2
  Downloaded num_cpus v1.16.0
  Downloaded gemm-f32 v0.17.0
  Downloaded reborrow v0.5.5
  Downloaded stable_deref_trait v1.2.0
  Downloaded rayon-core v1.12.1
  Downloaded seq-macro v0.3.5
  Downloaded thiserror-impl v1.0.56
  Downloaded dyn-stack v0.10.0
  Downloaded thiserror v1.0.56
  Downloaded unicode-xid v0.2.4
  Downloaded rand_chacha v0.3.1
  Downloaded ppv-lite86 v0.2.17
  Downloaded bytemuck_derive v1.5.0
  Downloaded getrandom v0.2.12
  Downloaded once_cell v1.19.0
  Downloaded unicode-ident v1.0.12
  Downloaded byteorder v1.5.0
  Downloaded crc32fast v1.3.2
  Downloaded num-complex v0.4.4
  Downloaded gemm-common v0.17.0
  Downloaded crossbeam-utils v0.8.19
  Downloaded quote v1.0.35
  Downloaded ryu v1.0.16
  Downloaded num-traits v0.2.17
  Downloaded zip v0.6.6
  Downloaded rand_distr v0.4.3
  Downloaded serde v1.0.195
  Downloaded rand v0.8.5
  Downloaded raw-cpuid v10.7.0
  Downloaded libm v0.2.8
  Downloaded serde_json v1.0.111
  Downloaded rayon v1.8.1
  Downloaded libc v0.2.152
  Downloaded gemm-c64 v0.17.0
  Downloaded gemm-c32 v0.17.0
  Downloaded safetensors v0.4.2
  Downloaded gemm-f64 v0.17.0
  Downloaded gemm v0.17.0
  Downloaded gemm-f16 v0.17.0
  Downloaded yoke v0.7.3
  Downloaded pulp v0.18.6
  Downloaded 60 crates (3.1 MB) in 14.91s
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   Compiling libc v0.2.152
   Compiling cfg-if v1.0.0
   Compiling libm v0.2.8
   Compiling autocfg v1.1.0
   Compiling crossbeam-utils v0.8.19
   Compiling ppv-lite86 v0.2.17
   Compiling rayon-core v1.12.1
   Compiling reborrow v0.5.5
   Compiling paste v1.0.14
   Compiling either v1.9.0
   Compiling bitflags v1.3.2
   Compiling seq-macro v0.3.5
   Compiling once_cell v1.19.0
   Compiling unicode-xid v0.2.4
   Compiling raw-cpuid v10.7.0
   Compiling serde v1.0.195
   Compiling crc32fast v1.3.2
   Compiling serde_json v1.0.111
   Compiling stable_deref_trait v1.2.0
   Compiling itoa v1.0.10
   Compiling ryu v1.0.16
   Compiling thiserror v1.0.56
   Compiling byteorder v1.5.0
   Compiling num-traits v0.2.17
   Compiling zip v0.6.6
   Compiling crossbeam-epoch v0.9.18
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling crossbeam-deque v0.8.5
   Compiling getrandom v0.2.12
   Compiling memmap2 v0.9.4
   Compiling num_cpus v1.16.0
   Compiling rand_core v0.6.4
   Compiling rand_chacha v0.3.1
   Compiling rayon v1.8.1
   Compiling rand v0.8.5
   Compiling rand_distr v0.4.3
   Compiling synstructure v0.13.0
   Compiling bytemuck_derive v1.5.0
   Compiling serde_derive v1.0.195
   Compiling zerofrom-derive v0.1.3
   Compiling thiserror-impl v1.0.56
   Compiling yoke-derive v0.7.3
   Compiling bytemuck v1.14.1
   Compiling num-complex v0.4.4
   Compiling dyn-stack v0.10.0
   Compiling half v2.3.1
   Compiling zerofrom v0.1.3
   Compiling yoke v0.7.3
   Compiling pulp v0.18.6
   Compiling gemm-common v0.17.0
   Compiling gemm-f32 v0.17.0
   Compiling gemm-c64 v0.17.0
   Compiling gemm-f64 v0.17.0
   Compiling gemm-c32 v0.17.0
   Compiling gemm-f16 v0.17.0
   Compiling gemm v0.17.0
   Compiling safetensors v0.4.2
   Compiling candle-core v0.3.3 (https://github.com/huggingface/candle.git#fd7c8565)
   Compiling myapp v0.1.0 (/home/pdd/myrust/myapp)
    Finished dev [unoptimized + debuginfo] target(s) in 32.90s
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

candle_test的简单测试项目

• https://github.com/RileySeaburg/candle_test

• git clone https://github.com/RileySeaburg/candle_test.git

Cargo.toml 文件

[package]
name = "candle_test"
version = "0.1.0"
edition = "2021" #  Rust 版本

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
candle-core = { git = "https://github.com/huggingface/candle.git", version = "0.2.1", features = ["cuda"] }
# `candle-core`：项目依赖的包的名称。`git` 字段指定了包的源代码仓库地址。`version` 字段指定了使用的包的版本。`features` 字段是一个数组，指定了启用的功能。在这里，启用了 "cuda" 功能。
# 可以通过以下命令添加，取消可注释掉"cuda"，再cargo build
# cargo add --git https://github.com/huggingface/candle.git candle-core
# cargo add candle-core --features cuda
1
2
3
4
5
6
7
8
9
10
11
12
13

main.rs

use candle_core::{DType, Device, Result, Tensor};

// 定义一个模型结构体
struct Model {
    first: Tensor,
    second: Tensor,
}

impl Model {
    // 定义模型的前向传播方法
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = image.matmul(&self.first)?; // 输入乘以第一层权重
        let x = x.relu()?; // 使用 ReLU 激活函数
        x.matmul(&self.second) // 结果乘以第二层权重
    }
}

fn main() -> Result<()> {
    // 初始化设备，如果 GPU 可用则使用 GPU，否则使用 CPU
    let device = match Device::new_cuda(0) {
        Ok(device) => device,
        Err(_) => Device::Cpu,
    };

    // 创建模型的第一层和第二层权重张量
    let first = Tensor::zeros((784, 100), DType::F32, &device)
        .unwrap()
        .contiguous()?;
    let second = Tensor::zeros((100, 10), DType::F32, &device)
        .unwrap()
        .contiguous()?;
    
    // 初始化模型
    let model = Model { first, second };

    // 创建一个用于测试的虚拟图像张量
    let dummy_image = Tensor::zeros((1, 784), DType::F32, &device)
        .unwrap()
        .contiguous()?;

    // 调用模型的前向传播方法获取预测结果
    let digit = model.forward(&dummy_image)?;

    // 打印预测结果
    println!("Digit {digit:?} digit");

    Ok(())
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

知识点总结

candle_core:: Result

// Result定义在/home/pdd/.cargo/git/checkouts/candle-0c2b4fa9e5801351/e8e3375/candle-core/src/error.rs
pub type Result<T> = std::result::Result<T, Error>; // 定义了一个 `Result` 类型，这是一个 `Result<T, Error>` 类型的别名。其中 `T` 是成功时的返回类型，而 `Error` 是失败时的错误类型。
1
2

// Ok(()) 定义在 /home/pdd/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs
// 这是 Rust 标准库中的 `Result` 公共的枚举类型，它有两个泛型参数 `T` 和 `E`。`T` 代表成功时返回的值的类型，`E` 代表错误时返回的错误类型。
// #[]是属性（attribute）,提供额外信息
pub enum Result<T, E> {
    /// Contains the success value
    #[lang = "Ok"]
    #[stable(feature = "rust1", since = "1.0.0")]
    Ok(#[stable(feature = "rust1", since = "1.0.0")] T),// `Ok(T)`: 这是 `Result` 枚举的一个变体，用于表示成功的情况
                                                        // (): 是 Rust 中的单元类型（unit type），类似于其他语言中的 void。

    /// Contains the error value
    #[lang = "Err"]
    #[stable(feature = "rust1", since = "1.0.0")]
    Err(#[stable(feature = "rust1", since = "1.0.0")] E),// `Err(E)`: 这是 `Result` 枚举的另一个变体，用于表示错误的情况。
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

?符号

• 在 Rust 中，? 符号用于处理 Result 或 Option 类型的返回值。这个符号的作用是将可能的错误或 None 值快速传播到调用链的最上层，使得代码更加简洁和易读。

fn forward(&self, image: &Tensor) -> Result<Tensor> {
    let x = image.matmul(&self.first)?; // 如果matmul返回Err，则整个forward函数返回Err
    let x = x.relu()?; // 如果relu返回Err，则整个forward函数返回Err
    x.matmul(&self.second) // 如果matmul返回Err，则整个forward函数返回Err；否则返回Ok(Tensor)
}
1
2
3
4
5

语句和表达式:语句以分号结尾，而表达式通常不需要分号。

• 函数体：函数体是一个块表达式，其值是最后一个表达式的值。

  fn add(x: i32, y: i32) -> i32 {
      x + y // 表达式
  }
  1
  2
  3

CG

• Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.
• resnet for caddle: https://github.com/iFREEGROUP/candle-models
• Using candle to build a transformers for Rust.
• https://github.com/joker3212/candle-clip
• https://github.com/jonysugianto/candle_fastformer
• Candle Silu inplace
• https://github.com/ansleliu/PortableTelemedicineMonitoringSystem
• https://mobile-aloha.github.io/
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarehttps://arxiv.org/pdf/2304.13705.pdf
• A flexible, high-performance 3D simulator for Embodied AI research.
• https://github.com/huggingface/huggingface.js
• https://www.zhihu.com/people/wasmedge
• https://wasmedge.org/docs/start/install/