Traveling plan for Iceland in Dec 2023. It is a pain to use Notion and Google Maps to plan the trip – Notion is too slow, compact and lumped together, and Google Maps is awkward to use for planning. Moreover, information is scattered everywhere, and it is really hard to find a good source of information and keep things organized.
A post of investigating the possibility of making my own ASIC chip. My initial thought was to design a custom Yubikey and use it as a shortcut key to awake AI agents; something like the iPhone 15 side button. Later I realized it might be more interesting to design a mini ASIC as a learning project, since I’ve never got a chance to get involved in the chip design process.
Bill Dally’s invited talk at Cornell. Bill is a senior big name in accelerator/CompArch since 1985. The talk primarily focuses on Nvidia’s efforts to enhance GPU performance, starting from Kepler in the early 2000s and extending to the recent Hopper H200, which includes improvements in low-precision and sparsity support. In addition to the technical aspects, an intriguing comment he made about AI caught my attention. It directly echoes Rich Sutton’s “Bitter Lessons” blog, emphasizing that AI model architectures remain simple, much like they were 50 years ago; the only significant change lies in the increased compute power at our disposal, making it possible for us to stack deeper layers. Modern hardware enables LLM-like Bayesian-based AI models to more effectively approximate the true distribution of the world.
Basic concepts in ML/AI algorithms, and some simple algorithms implementations. There are so many operators and variants of them; this post only covers the most basic ones.
Watched the final chapter of Attack on Titan (AOT) in double speed. Setting aside the emotional parts, there are a few things that left a deep impression on me. Firstly, the counterattack against nihilism: humans are just a material entity with the attribute of ‘procreation,’ and the so-called freedom is false. The cyclic nature of the game is repetitive and meaningless, yet the process of existence brings joy. However, this sense of small happiness, or existentialism, is merely shifting the center of coordinates from the universe to oneself. Another thing is conflict. Strife and combat exist among all living beings, so the Voluntary Human Extinction Movement (VHEMT) is a false proposition. Even if humans disappear, other life forms will continue to engage in conflict and destruction. Heroes may step forward to bring an end to war, but war itself will not cease.
Classic compiler frameworks such as Clang, LLVM, MLIR, etc. AI optimizing compilers like Torch, TensorRT, TVM, XLA/HLO, JAX/Pallas, Triton, etc. Also includes wider topics such as programming models, and program synthesis techniques, such as High-Level Synthesis (HLS), target-specific domain specific languages, etc.
AI hardware accelerators, including FPGA, and other emerging ASICs like Google TPU, AMD AIE, AWS Trainium, etc. This note does not include details about the software stack, but only focuses on the hardware architecture and its design choices.
Basic concepts in ML/AI algorithms, and some simple algorithms implementations. There are so many operators and variants of them; this post only covers the most basic ones.