• In general, it is best to test smaller portions of your program as you work, to make sure each part is working right.
• In general, you can use a command like print in Python or disp in Matlab to display variable values in the middle of program execution --- that can help with debugging if you need it.

For neural networks:
If you change your nonlinearity from sigmoid, you should really adjust your gradient descent rule to reflect the new nonlinearity
(eg, rederive d/dw (y - g(w^T x+b))²)

For SVM:
We never established the gradient descent rule for SVM (the rule shown in the SVM slides discussing hyperparameters is the update rule for logistic classification). You should figure out the update rule based on the derivative of the formula given in the project write-up. It actually would be best to take a derivative for the γ as well (learn the γ) and even learn a separate γ for each data point (0 γs would be like 0 αs, indicating non support vectors)
min_i,γ w^Tw + Σ_{i in +1}γⁱ(1-(wTxⁱ+b)) + Σ_{i in -1}γⁱ(w^Txⁱ+b + 1) + Σ_{i in All}γⁱ

(You can also use the simpler version given in the write-up (probably will classify less well):
min_i,γ w^Tw + γΣ_{i in +1}(1-(wTxⁱ+b)) + γΣ_{i in -1}(w^Txⁱ+b + 1)
)