This paper demonstrates successful Differential Power Analysis (DPA) attacks on an FPGA-based binarized neural network classifier to extract the weights and biases. It also proposes effective masking and hiding-based countermeasures to thwart such attacks.