I'd like to show off MIND, (GitHub - dev-null321/MIND: Deep learning library) which is a deep learning library I wrote in Racket. It can be used to make simple FNN systems. The library includes essential components such as tensor operations, activation functions, loss functions, and back-propagation for gradient calculation and weight updates. It's currently in it's infancy so it is lacking a lot of things that a great DL library would really need. So currently there is no support for a CNN system. Other things that need to be added are different kinds of activation functions, support for other DL architectures, optimizations, optimizers, etc.
I hope the community enjoys this, and that future improvements can be made.
I'll be releasing a new update , with a test file for xor within the coming week. It will also include sigmoid, tahn, and a few other improvements. After this I plan to rewrite most of the tensor creation and matrix multiplication in C. The only hang ups are getting the tensor sizes to be correct and crushing all the bugs i'm occurring. More to come.
Great!
It should be interesting to compare it to malt (from the Little Learner).
While on the subject, I wonder if it would make sense to have a standardised, efficient implementation of tensors, available for everyone to use, rather than each new DL library having its own. Just a thought...
A standardization for tensor implementation is an interesting idea, something I've never actually thought about. Please let me know what you think of the library, I appreciate any and all feedback!
I certainly will, though it might take a while as I'm fairly new to all this and picking it up as fast as I can.
Yeah, I think a standardised tensor library would not only be great for whatever optimizations could be made on it, but also as a fundamental starting point for beginners who want to understand the nature of the beast, try basic algorithms, etc. I don't mean a full DL library, just an API that exposes the tensor implementation and its interfaces. I have been enjoying learning with malt, but it does hijack the whole Racket numeric stack and I'm not sure how I feel about that. Sometimes it's great, sometimes it's not.
I think the systems should coexist and be orthogonal, so that I can invoke them from any other software I choose.
Looks useful, but I'm not sure it covers what I mean.
Before I got introduced to tensors in the AI context I had played around a bit with J (Iverson's APL spin-off) so I was reasonably comfortable with notions of tensor rank, multiplications at different ranks and so on. I'm certain I'd be struggling a lot more if I hadn't had that exposure. That was the motivation behind my comment.
An established library for this could mean that those who care about such things could make the best optimizations (e.g. using modern processors' features available for this kind of thing) and everyone else wouldn't go re-inventing the wheel all the time. That's all.
