# Announcing The Little Learner: A Straight Line to Deep Learning **URL:** \ **Category:** General\ **Tags:** book\ **Created:** [September 7, 2022, 10:42pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288 "2022-09-07T22:42:16Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![themetaschemer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/themetaschemer/32/718_2.png) [@themetaschemer](https://racket.discourse.group/u/themetaschemer)\ **Post date:** [September 7, 2022, 10:42pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/1 "2022-09-07T22:42:17Z")
Daniel P. Friedman and I (Anurag Mendhekar) are pleased to announce that our upcoming book The Little Learner: A Straight Line to Deep Learning just got its release date, complete with a Preorder Sale [(Barnes and Noble, 25%)](http://www.barnesandnoble.com/w/?ean=9780262546379) The book comes out on 2/21/2023. "The Little Learner" covers all the concepts necessary to develop an intuitive understanding of the workings of deep neural networks: tensors, extended operators, gradient descent algorithms, artificial neurons, dense networks, convolutional networks, residual networks and automatic differentiation. The authors aim to explain the workings of Deep Learning to readers who may not have the mathematical sophistication necessary to read the existing literature on the subject. Unlike other books in the field, this book makes very few assumptions about background knowledge (high-school mathematics and familiarity with programming). The authors use a layered approach to construct advanced concepts from first principles using really small (“little”) programs that build on one another. This is one of the things that makes this book unique. The other is that it introduces these ideas using a conversational style in Question/Answer format that is characteristic of the other books in the Little series. The conversational style puts the reader at ease and enables the introduction of ideas in frame-by-frame manner as opposed to being hit with a wall of text. It is (of course!) written using elementary Scheme and the code will be released as a Racket package. ![TLLPresale](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/6a7a4a89896b6706531a376782b7ca156efa8e7a.jpeg) ---
**Author:** ![spdegabrielle](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/spdegabrielle/32/95_2.png) [@spdegabrielle](https://racket.discourse.group/u/spdegabrielle)\ **Post date:** [September 8, 2022, 7:54am UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/2 "2022-09-08T07:54:34Z")
Thank you @themetaschemer :racket_heart: I’ve added it to [Books · racket/racket Wiki · GitHub](https://github.com/racket/racket/wiki/Books) Is there a publisher page I can link to? Best regards Stephen :beetle: ---
**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\ **Post date:** [September 8, 2022, 8:37am UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/3 "2022-09-08T08:37:43Z")
This is awesome! Are you able to share the table of contents by any chance? ---
**Author:** ![peat](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/peat/32/505_2.png) [@peat](https://racket.discourse.group/u/peat)\ **Post date:** [September 8, 2022, 1:42pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/4 "2022-09-08T13:42:32Z")
I have to wait until Feb 23? The cruelty! VERY excited to read this—I’ve wanted something like this for a long time. If you want an early reader with a mind unspoiled by any particular intelligence or preexisting deep understanding of the domain, I offer up mine. 😉 ---
**Author:** ![themetaschemer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/themetaschemer/32/718_2.png) [@themetaschemer](https://racket.discourse.group/u/themetaschemer)\ **Post date:** [September 8, 2022, 2:48pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/5 "2022-09-08T14:48:28Z")
Thanks Stephen! The MIT Press hasn't put up a page for it yet, but both Amazon and BN have. ---
**Author:** ![themetaschemer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/themetaschemer/32/718_2.png) [@themetaschemer](https://racket.discourse.group/u/themetaschemer)\ **Post date:** [September 8, 2022, 3:06pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/6 "2022-09-08T15:06:09Z")
Laurent, our ToC by itself is a bit cryptic, but should give you some idea. The sequencing goes like this: Minimal Scheme Intro for those who don't know it; Minimal machine learning by hand; Tensors; Operator extension; Gradient descent; Stochastic Gradient descent and variations; Neurons + Universal approximation; Structuring Neural networks; Classification using dense layers; Signals; Convolutional layers; and two appendices that are entirely dedicated to Automatic Differentiation. ![Screen Shot 2022-09-08 at 7.49.21 AM](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/82fb9d7d76be680b88327edc7603d74bb4d6bf73.png) ---
**Author:** ![themetaschemer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/themetaschemer/32/718_2.png) [@themetaschemer](https://racket.discourse.group/u/themetaschemer)\ **Post date:** [September 8, 2022, 3:07pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/7 "2022-09-08T15:07:36Z")
Thanks, Pete! Will keep that in mind! ---
**Author:** ![jesse](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jesse/32/446_2.png) [@jesse](https://racket.discourse.group/u/jesse)\ **Post date:** [September 12, 2022, 12:05pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/8 "2022-09-12T12:05:29Z")
This will _definitely_ be on my bookshelves come winter/spring next year. Wonderful! ---
**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\ **Post date:** [February 25, 2023, 9:10pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/9 "2023-02-25T21:10:34Z")
Hey! It looks like the book is out! I'm super excited to buy a copy. So now, the hard part: whom to order it from? The vendors listed on the MIT Press website are ... heavily british? They list [amazon.co.uk](https://lmgtfy.app/?q=amazon+is+evil), Blackwells, [Bookshop.org](http://Bookshop.org), Foyles, Hive, and Waterstones. Any opinions? Um, aside from "not Amazon", which is pretty much my first and only criterion? ---
**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\ **Post date:** [February 25, 2023, 9:32pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/10 "2023-02-25T21:32:59Z")
Oh! A little investigation suggests that [bookshop.org](http://bookshop.org) is more or less exactly what small-business zealots like me are looking for. As a side note, it's absolutely _tragic_ that there isn't a single independent new-book dealer in this town of 54K people. ---
**Author:** ![spdegabrielle](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/spdegabrielle/32/95_2.png) [@spdegabrielle](https://racket.discourse.group/u/spdegabrielle)\ **Post date:** [February 26, 2023, 1:20am UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/11 "2023-02-26T01:20:13Z")
Any local bookseller should be able to order. If you are affiliated with a educational institution with a library ask if they can purchase it for you - it doesn’t matter if your are staff or student. This may also work for community libraries. This also applies to the other fine books listed at [https://racket-lang.org/books.html](https://racket-lang.org/books.html) Some directories of booksellers: - [Search the ABA Bookseller Member Directory | the American Booksellers Association](https://www.bookweb.org/member_directory/search/ABAmember) - [Booksellers Association - Bookshop Search](https://www.booksellers.org.uk/bookshopsearch) - [https://www.bookpeople.org.au/find-a-bookshop](https://www.bookpeople.org.au/find-a-bookshop) - [Find a Bookseller : Booksellers NZ](https://www.booksellers.co.nz/find-bookshop) - [Canadian Independent Booksellers Association - Find a Bookseller](https://cibabooks.ca/bookseller-map) ---
**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\ **Post date:** [February 26, 2023, 1:46am UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/12 "2023-02-26T01:46:36Z")
> [@jbclements](#): > > So now, the hard part: whom to order it from? … Any opinions? I'll put in a plug for the Seminary Co-op, "the country’s first not-for-profit bookstores whose mission is bookselling." It's on special order right now (which only adds a few days), but they're very likely to stock it if people order it. (They have actual human booksellers empowered to make decisions about books that seem interesting.) [https://www.semcoop.com/ingram-0?isbn=9780262546379](https://www.semcoop.com/ingram-0?isbn=9780262546379) ---
**Author:** ![jesse](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jesse/32/446_2.png) [@jesse](https://racket.discourse.group/u/jesse)\ **Post date:** [February 26, 2023, 8:21am UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/13 "2023-02-26T08:21:49Z")
I pre-ordered the book a couple months ago on [Amazon.de](http://Amazon.de). I got a pleasant surprise a few days ago when they charged my credit card, which they do only when they ship the book. Very much looking forward to diving in! ---
**Author:** ![dstorrs](https://avatars.discourse-cdn.com/v4/letter/d/898d66/32.png) [@dstorrs](https://racket.discourse.group/u/dstorrs)\ **Post date:** [March 15, 2023, 9:07pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/14 "2023-03-15T21:07:39Z")
Got my copy yesterday! Very excited. ---
**Author:** ![joeld](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/joeld/32/38_2.png) [@joeld](https://racket.discourse.group/u/joeld)\ **Post date:** [March 16, 2023, 3:06pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/15 "2023-03-16T15:06:43Z")
Just a note, I get a bad cert domain error when trying to use the link for semcoop. ---
**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\ **Post date:** [March 27, 2023, 3:53pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/16 "2023-03-27T15:53:09Z")
Whatever the problem was, it seems to have been fixed. ---
**Author:** ![dyoo](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/dyoo/32/1023_2.png) [@dyoo](https://racket.discourse.group/u/dyoo)\ **Post date:** [March 9, 2024, 8:48pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/17 "2024-03-09T20:48:00Z")
Some personal notes as I read it. It's a good book so far, but there are occasionally part of it that frustrate me to no end. It's the kind of book in which I'll probably start writing notes into the margin. Thankfully, there's a lot of room. I'm just past Chapter 5. One of the key problems I've had with it is really due to my own blasted curiosity. I was curious to see what would happen if I bumped up the number of `revs` from the tiny `1000` to something ridiculously larger, like `1,000,000.` That is, I should be able to do something like: ```scheme (with-hypers ((alpha 0.001) (revs 1000000)) (gradient-descent ((l2-loss plane) plane-xs plane-ys) (list (tensor 0.0 0.0) 0.0))) ``` and I expect this to take some time, but it should just work, right? But alas, no! And it doesn't work for non-obvious reasons. The representation of tensor values are using a special encoding that's used for automatic differentiation, this has the implication that the more operations we do in iterating a value (such as the theta model value), the more space the tensor representation is taking, and eventually we run out of memory. It's the same problem I ran into when working with rational numbers: by maintaining perfect precision, we can end up with ridiculous bignum numerators and denominators if we're not careful. After fighting the documentation, I figured out that a way to handle this is to rip out the special automatic-differentiation representation, the duality, like this: ```scheme ;; rip out the duality (define rip (ext1 ρ 0)) ``` After which, we can redefine gradient-descent to rip out duality between each iteration: ```scheme (define gradient-descent (lambda (obj theta) (let ([f (lambda (big-theta) (map rip (map (lambda (p g) (- p (* alpha g))) big-theta (gradient-of obj big-theta))))]) (revise f revs theta)))) ``` And now it works. So now I have a little more understanding on what's going on underneath the surface, and it makes me simultaneously happy and sad about it. Happy that I understand this better now, but sad that I am incomprehensible; there are very few people I interact with that care about this sort of stuff... Time to read more. ---
**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\ **Post date:** [March 10, 2024, 10:00pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/18 "2024-03-10T22:00:11Z")
I got stuck in this book earlier than you, your post makes me want to get back into it, many thanks for the bump! ---
**Author:** ![jjsimpso](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jjsimpso/32/602_2.png) [@jjsimpso](https://racket.discourse.group/u/jjsimpso)\ **Post date:** [March 11, 2024, 1:26pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/19 "2024-03-11T13:26:04Z")
Does the memory consumption issue still occur when using one of the alternate representations of tensors? I haven't tried anything other than the `'learner` representation so far. ---
**Author:** ![dyoo](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/dyoo/32/1023_2.png) [@dyoo](https://racket.discourse.group/u/dyoo)\ **Post date:** [March 11, 2024, 5:53pm UTC](https://racket.discourse.group/t/announcing-the-little-learner-a-straight-line-to-deep-learning/1288/20 "2024-03-11T17:53:30Z")
I did try switching representations to `'flat-tensors`, but it did not seem to help. It's possible that I didn't do it right. Independent confirmation would be interesting and useful to hear. I finished chapter 8 over the weekend. This was one of the more frustrating chapters to me, but not because the material was difficult, but because the analogy of momentum to relay runners did not feel natural. It felt so contrived that it really bothered me for the rest of the chapter, honestly. (I suspect the miss for me here may be a cultural thing: I do not have any particular fondness to Groucho, Chico, Harpo, Gummo, or Zeppo.) I think I can summarize the last few chapters: - Chapter 6: add batching with random selection so we can do stochastic gradient descent. - Chapter 7: generalizes gradient-descent so that each parameter can be augmented, or "accompanied" by additional data to carry state over the iterations. The expected changes are to use functions to (1) wrap each parameter "inflate" on entry, (2) unwrap each parameter before exit "deflate", and finally, (3) update each parameter, incorporating the extra state. Since "inflate", "deflate", and "update" share the same suffix, this chapter jokes about this a bit (perhaps a bit too much). - Chapter 8: uses the results from chapter 7 and shows how to attach auxiliary "velocity" to each parameter. We add a momentum mechanism during gradient update, to help smooth the walk through the parameter space. External resources like: [Momentum - Cornell University Computational Optimization Open Textbook - Optimization Wiki](https://optimization.cbe.cornell.edu/index.php?title=Momentum#Problems_with_Gradient_Descent) better explain what's happening when we introduce momentum: the incorporation of momentum helps to reduce oscillation in practice.