training on mnist (old)

got the mnist in csv format from https://www.kaggle.com/datasets/oddrationale/mnist-in-csv im gonna use a feedforward neural network: we load the dataset using the cl-csv library

(ql:quickload "cl-csv")
(defparameter *train-data* (cl-csv:read-csv (pathname train-data-file)))
(defparameter *test-data* (cl-csv:read-csv (pathname test-data-file)))
(defparameter *train-data* (mapcar (lambda (mylist) (mapcar #'parse-integer mylist)) (cdr *train-data*)))
(defparameter *test-data* (mapcar (lambda (mylist) (mapcar #'parse-integer mylist)) (cdr *test-data*)))

*TEST-DATA*

next we create the network, each image is 28x28 pixels so we need an input layer of that size, and an output layer of size 10 because it needs to output 0-9

(defparameter nw (make-network
                  :input-layer-size (* 28 28)
                  :hidden-layer-sizes '(10)
                  :learning-rate 0.01
                  :output-layer-size 10))

with this network we have weights to update for each training example, its a large network and would take alot of time since we're only using a single CPU thread

(defun test ()
  (network-train
   nw
   (list->vector (mapcar 'cdr *train-data*))
   (list->vector (mapcar (lambda (digit)
                           (let ((mylist (make-list 10 :initial-element 0)))
                             (setf (nth digit mylist) 1)
                             mylist))
                         (mapcar 'car *train-data*)))))

i was getting (relatively) good results even after a few epochs