The array.batch.size parameter

We only had 400 instances (rows) in our data, which can easily fit into memory. However, if your input data has millions of instances, the data needs to be split into batches during training in order to fit in the memory of the CPU/GPU. The number of instances you train at a time is the batch size. Note, you still iterate over all the data for the number of epochs, you just split the data into batches during each iteration and run the forward-propagation, backpropagation step over each batch for each epoch. For example, if you had 100 instances and selected a batch size of 32 with 6 epochs, you would need 4 batches for each epoch (100/32 = 3.125, so we need 4 batches to process all the data), for a total of 24 loops.

There is a trade-off in choosing the batch size. If you choose too low a value, the model will take a longer time to train because it's running more operations and batches will have more variability because of the small size. You cannot choose an enormous batch size either, this might cause your model to crash because it loads too much data into either the CPU or GPU. In most cases, you either take a sensible default that works from another deep learning model, or you set it at some value (for example, 1,024) and if your model crashes, then try again with a value of half the previous value (512).

There is a relationship between Batch size, Learning rates, and # Epochs for training. But there are no hard and fast rules in selecting values. However, in general, consider changing these values together and do not use an extreme value for one of these hyper-parameters. For example, picking a large Learning rate should mean fewer epochs, but if your batch size is too small, the model may fail to train. The best advice is to look at similar architectures and pick a similar set and range of values.

Now that we can see that deep learning still uses many of the concepts from neural networks, we will move on to talk about an important issue that you will probably encounter with every deep learning model: overfitting.