I like the idea of applying the concept of "Crossing the Chasm" to teaching. It was originally coined by Steve Blank to describe marketing high tech products.
It is my job as a teacher to meet the student "where they are", but in order to do that I have to cross the Chasm first and then cross back. This is generally the skill that differentiates skilled teachers and curriculum developers from subject matter experts.
In the context of deep learning, the challenge is complicated by the fact that the prerequisite material is so cross discipline that no two students have the same background.
If you are interested in getting feedback about where you stand right now, I encourage you to take this diagnostic and we can go from there.
My Second Attempt at AI-assisted Hardware / Software Co-Design
May 20, 2023
by Steve Goldsmith
I'll show rather than tell:
In order for this to be a reasonable approach I need a massive force multiplier. Based on these two attempts at code generation using phind, I
feel pretty confident we are almost there. But I am still going to cling onto the late mover advantage a while longer.
This paper is a great call to action, but I think it's missing something bigger.
With papers like this one coming out seemingly every day, I think the era of generating machine code and silicon directly from LLMs
is arriving sooner than I would have thought.
I'll let you draw your own conclusions, but for the time being I am going to steer clear of open silicon and focus on proper MPWs using commercial EDA tools and plan on using service providers like
Muse, Europractice, and CMC.
- Git
- Style Guides
- Debugging
- Refactoring
- Documentation
- Testing
- Pairing
- Zen of Python
- Convention vs. Configuration
- Principle of Least Astonishment
- Open Source Contribution Project
Textbook: Code Complete: A Practical Handbook of Software Construction, McConnell, Steve
Textbook: The Pragmatic Programmer: Your Journey to Mastery, Thomas, David; Hunt, Andrew
Machine Learning (Python/TensorFlow)
- Python Refresher (optional)
- TensorFlow Basics
- Layers
- Parameters
- Training
- Inference
- Models
- Transfer Learning
- Machine Vision
- Natural Language Processing
- Tensors
- ML Hardware
- Image Classifier Project
Computer Science (Python/C/Javascript)
- Graphs
- Trees
- Hash Tables
- Big O Notation
- Computational Complexity Classes
- Sorting
- Search
- Intro to Set, Order, and Category Theory
- Chomsky Hierarchy
- Web Scale Project
- Clock Gating
- Clock Domain Crossings
- Sychronizers
- Async FIFOs
- Frequency Scaling
- Microarchitecture for Low Power
- System Architecture for Low Power
- Low Power ASIC Project
SoC Design (PSPICE/Verilog/Python)
- RISC-V System Model
- Memory Map Design
- Bus Design
- Cache Hierarchy Design
- I/O Design
- Floorplanning
- Clock Domain Design
- Multiprocessor Systems
- Network-on-Chip
- SoC Project
Semiconductor Tech Seminar: Overview of Post-Denard Scaling Era Technologies
A lot of the prose itself that went with these diagrams needs to be pulled from long email threads with various embedded system and semiconductor folks, but for now here are some random diagrams.
This is probably the most important diagram. My current take on co-design.
Here's where I think Aurifex Labs fits into the ecosystem. Everybody except Aurifex Labs are just examples.
One part of co-design I see emerging is "putting stakes in the ground" or more like pitons. Here's a HW piton for IoT:
I helped with the tapeout of a systolic array last fall. Here's the beating heart of a TPU:
Zooming in...
The CPU that this is the control unit for has been lost to the sands of time, but you can almost see the microarchitecture it if you look hard enough.
After much consternation I have decided to simplify the course offerings and focus on AI.
Here are the major changes:
- Class size is now 3 instead of 5
- Courses have fixed start and end dates
- Stopped offering the Web Development progam and Intro to Python course
- Consolidated ML into a single course
These changes are based on an increased interest and personal focus on ML combined with learning from experience that splitting up the ML content into multiple courses didn't really make sense.
Learning a new progamming language often changes the way we think about programming. Sometimes though a language will
change the way you think about hardware too. Forth does both.
Forth is a stack based language, but I am going to focus on using it as a minimal language that can be implemented in a few days.
I created a Forth demo to see it running in the browser.
To understand Forth I would start by reading the
jonesforth source (literate CISC at it's finest)
And here's the source for my little (incomplete) Forth including the emulator for the ad hoc assembly language.
My (Reductionist) View of Existential AI Risk and Regulation
April 15, 2023
by Steve Goldsmith
I think there are some AI risks that are more immediate than others and
things like bias and homogeneity are important, but I want to start with
a simple mental model of a couple of broad classes of existential risk. I don't know
if this is the right mental model, but it led to some interesting thoughts so here it is.
In the Emergent Takeover scenario, all actors in the universe may even be benevolent
except a single artificial actor that simply due to it's size has emergent
behaviors akin to survival instincts, but frequently emerge as an
existential risk to humanity in an AI system of a certain size. This system is probably more heterogenous and
complex (or complicated) than just an LLM. You could also call this the Skynet scenario.
In the Evil Attacker scenario, a human actor whose motivation is to do harm
uses a superintelligence as a force multiplier. It could be a single high net
worth individual whose mental health has been comprimised or a nation state
using it as a nuclear-style deterrent even if designed to go unused, but
what's in common is a purposeful and determined effort with significant
resources to cause existential harm or the threat of existential harm.
Think Bond villian.
I might have the ordering or spacing of the circles wrong. And there are interactions between
risks that influence society in more subtle ways and these risks. At this point, I really have no idea, but it feels like it's something and at the same time feels like it's the result of a mind overfitted to movies.
Future posts will likely deal with more concrete aspects of ML. I wanted to start with the existential and drill down.
by Steve Goldsmith
