AI Algorithms and Operators
Basic concepts in ML/AI algorithms, and some simple algorithms implementations. There are so many operators and variants of them; this post only covers the most basic ones.
Basics
- Convolution
- Vanilla 2D convolution: padding is fused into the kernel, and we map the output coordinates to the input coordinates to decide which input elements to use.
for (index_t b = 0; b < batch; ++b) {
for (index_t oc = 0; oc < out_channel; ++oc) {
for (index_t oh = 0; oh < output_height; ++oh) {
for (index_t ow = 0; ow < output_width; ++ow) {
data_t sum = 0;
// Loop over input channels, kernel height, and kernel width
for (index_t ic = 0; ic < in_channel; ++ic) {
for (index_t kh = 0; kh < kernel_height; ++kh) {
for (index_t kw = 0; kw < kernel_width; ++kw) {
// Calculate input indices
index_t ih = oh * stride + kh - padding;
index_t iw = ow * stride + kw - padding;
// Check if indices are within bounds
if (ih >= 0 && ih < height && iw >= 0 && iw < width)
sum += input[b][ic][ih][iw] \
* kernel[oc][ic][kh][kw];
}
}
}
// Write the result to the output
output[b][oc][oh][ow] = sum;
}
}
}
}