<Critical Notes>
- X / Y shape
	- X can be (x1) or (x1, 1) if given a single col
	- Y (if a single col) has to be (y1)
		- Likely a problem with scoring functions!

<To Do>
- Add a 'draw net' feature that plots a diagram of the net in pyplot
	- Normal (line) diagram
	- Heatmap visual of weights
- Backwards propogation? (Train net from x -> y; can the weights be reversed to go y -> x?)
- Language processing
	- Letters to slope (for a sorta 'sentiment' value)
	- Given a question, have an output for yes/no
- (!) Add better PATH handling when saving/loading models
- Check if validation-training makes sense for these models
	- Test nets training off of 'validation' subset made at each iteration
	- Keep history of training scores and validation scores in .fit
- (!) Use HuggingFace's package 'dataset' for standard model testing
- Make ensemble/cross-validation class of MCRegressors
- Make classifiers from regressors (output activation as 'sig' or something)
- Test TweakWeight selection kwarg as "Dropout" alternative
- Test models as an over-fitting memory cell
- Change Verbose to update every # of iterations if > 0


V1.5.0 (Speed! Models! Automake Models!)
- Added a check in .predict to check if model is fitted before complaining about self.inSize != # of X features
- Removed [-1, 1] clip when reported r2d2 score in .score (this removes the need to train specifically to SSE first)
- Changed .fit to return the score history
- Changing the predictions in .predict to a numpy array AND reshaping them in the same line screwed up model results
	- Seperated them into two lines now and it works just fine lmao????
- .fit description changes
	- Removed 'net' description (returns in-place now)
	- Added 'mre' (and to .score)
	- Changed "returns trained model" to "returns score history"
	- Removed old desc. about Beta=0 behavior (dynamic)
	- Added useMP description
- (!) Added Multiprocessing function in .fit
	- automatically kicks in when Beta*len(X) >= 4200
- (!) Changed default MCRegressor size to [100,] (yay!)
- (!) Changed default SUNN size to [100,] (yay2!)
- Added Multiprocessing thingy to SUNN.fit
- Added .structured attribute to SUNN
	- Accounted for it in .tweakSUA, .setSUAs, and .copy_su
- Deprecated fastCalc
- (!) Improved Calculate speed for MCRegressor (by factor of 5!)
	- Basically now just uses np.sum instead of sum
	- Condensed lines by favoring usage of np.dot
- (!) Improved Calculate speed for SUNN
- (!) Apparently MP not that helpful now, set default useMP to False in both .fits
	- The lower limit of # of calculations for MP being better might be higher
		- (At least on laptop tests anyways)
	- On laptop, lower limit for equal times is around: Beta*len(X) >= 110,000
	- Changed limit in .fit to reflect 
- (!) Renamed "MCRegressor" to "MCNeuralNetwork"
- (!) Added cross_val()
- Added .load()
- Added save_model()
- Renamed loadMN to load_model()
- Added build_model() to old section
	- Was a good test, but round-robin testing of different activation functions
	  will likely be better than how this one works.
- (!) Added MCSimpleEnsemble
	- Not many options as-is, but allows a simple generation of multiple of the same base model
	  trained to some data to act as a (you guessed it) simple ensemble
- (!) Added generate_model
	- Tests model activation combinations combining a greedy and round-robin style algorithm
	- Converges to useful solutions much quicker than previous methods
	- Changed train_split_percent to be low (fast and still does good)
	- Added option to skip_current iteration if its testing the current layer's AF in the model
- (!) Added biases to MCNeuralNetwork!
	- Have a few sneaky attributes that can be altered:
		- use_biases:BOOL
			- Allows for the model to change/use biases
			- the biases are initallized as zeros, therefore, changing
			  this before calling .fit prevents a model from using biases
		- bias_bounds
			- Allows for changing the min/max weight values of the biases
- (!) Added copy as an import, used for copy.deepcopy() in CopyNet()
	- Adding new attributes to copy has been getting annoying lol
- (!) Deprecated debug_calculate
	- useFast now doesn't change anything
	- Will keep the kwarg here for future use if another faster calculate method is made 
- Added a sneaky weights_bounds attribute
	- So far doesn't seem to help the model's fit at all, so [-1, 1] default is still good
- Renamed SUNN.copy_su to SUNN.CopyNet
	- Calls the MCNeuralNetwork .CopyNet now (using deepcopy)
- (!) Fixed r2d2 to flatten yHat array if yTrue is also flat (one dim)
	- Changed to just reshape yhat to ytrue shape
- (!) Added SOUP Model (Regressor and Classifier)
	- Basically a giant function per-feature that fits coefficients for each term in the function
- (! for me) Added run_module_tests to preform a standard check of model functionality before making a Released
	- Things fixed this time:
		- Changed kwarg 'hiddenCounts' to hidden_counts in SUNN for consitency
		- Added 'CopyNet' (change name later for all models!) to SOUP models
		- Added **kwargs to .score for SOUP so method=... doesn't affect it


V1.4.2 (Activation Optimizer!)
- Renamed X to xArr in .predict
- Added .optimize_fit
	- Finds optimal AF set for a model and given X/Y set
	- Fixed testing input/output activations not giving best results (kept changing xt, xv, yt, yv data each iteration)
	- Added usage of all four score types in .score method
- Fixed running model first-time fit setup only once
	- Added .fitted attribute
- Fixed .fit reporting Y features are "1 and not 1" when given a shape like (100, 1) and not (100,)
- Fixed .predict giving a 3D array when the third dimension (length of 3) was not needed and created casting errors later on
- Fixed .score not using .predict for both MCRegressor and SUNN types
	- .predict_su was used in netMetrics which is how it got There
	- .predict_su is now not needed and is deleted (was identical to .predict)
		- Only change was used .calculate_su instead of .Calculate (but SUNN.Calculate now points to .calculate_su)


V1.4.1 (Misc Cleaning)
- SQR AF gives either 0 or 1 now, not -1 or 1
- Updated init description
- Tested new GitHub quick start code


V1.4.0 (fit/predict, auto detection of # of features, saving method) - Released
- (!) Renamed "AdvNet" to "MCRegressor"
- (!) Renamed ".Fit" to ".fit"
- (!) Renamed ".Predict" to ".predict"
- Deprecated/moved/renamed .SaveNN to oldSaveNN
- Deprecated/renamed LoadNet to oldLoadNet
- (!) Added "save" method using joblib for MCRegressor
- (!) Added "loadMC" function using joblib for loading model objects previously saved
- (!) # of Input and Output features are now determined automatically
	- Moved inSize/outSize generation to .fit
	- Moved weight set-up to .fit (needs size info)
	- Set .parameters to "Not yet generated; call .fit first" until weights are generated
	- Moved net speed calculation to .fit; is "Not yet generated; call .fit first" at first
	- Weight medians in __str__ show "Not yet generated; call .fit first" before being fit
	- Updated .CopyNet to move/generate the model parameters correctly from old way
- Changed TTSplit to return in order xTrain, xTest, yTrain, yTest (was xTrain, yTrain, xTest, yTest)
- Changed TTSplit parameter percentTrain default to 70 
- Deprecated ExpTrain (for now!)
- Set default score functions in .fit for lower is better & to use SSE by default
- (!) Finally made .fit to return in place (was working, just had to fix scoring objectives stuff)
	- Could be worth seeing if weights & _weights property stuff is needed, but it actually
	  is kinda of nice to have automatically copy the weights over so I'll keep is
- Deprecated Forecast
- Updated __init__ description for MCRegressor with examples
- (!) External activation functions are now allowed (either given alone or in a list)
- (!) Activation functions are now made into lil defined functions and can be called from
  the dictionary in applyActi (This means match/case stuff won't be needed!!)
- There are now four score types ['r2', 'sse', 'mae', 'rae'] in netMetrics and that can be
  used in .fit as well.
	- .fit will automatically switch zeroIsBetter for 'r2' and 'rae' score types
- .fit now displays which score type is being used when Verbose is True, along with the 
  other training progress information
- (!) matplotlib.pyplot removed as dependancy/useage in main
- (!) Added support for Pandas DataFrames and Series
	- Still new and should be considered in beta but seems to work fine!
- (!) Added SUNN model 
	- Most definitley still in beta!!!
	- Allows for using a different activation function for every node
	- Might be useful as an optimal AF-per-layer-finder 
	- Added option for force input/output layers to have a certain AF
- Activation Function Work:
	- Changed ATAN to TANH (faster)
	- Added correct sigmoid
	- Added SILU
	- Added dSILU
	- Updated MCRegressor init description
- Default MCRegressor activations gives lin/silu/.../lin
- Moved netMetrics into .score method


V1.3.1 (PyPI Page)
- Removed quickstart code example from PyPI README (didn't format right)


V1.3.0 (Extra AF layer, r2d2 work)
- netMetrics uses r2d2 now in calculation
- Removed "return all" option for r2 things in old netMetrics calculation
- Confirmed r2d2 and netMetrics both return equal values as SKLearn.metrics.r2_score
- Added np.clip to netMetrics as TEMPORARY cleanup of negative values
- Removed sse, aae, mae options from netMetrics 
- Updated calculation method in full .Calculate to account for first AF layer
	- Updated fastCalc, put a copy of old method under old stuff
- Updated __init__ check of correct number of AFs (now is equal to len(size) not len(weights))
- Clarrified size inputs to AdvNet (# input features, hidden layer heights, # output features)


V1.2.1 (r2d2 hotfix)
- Fixed r2d2 equation used
- Added type checking to r2d2


V1.2.0 (Renaming/Cleaning)
- Added .Predict()
- Added 'root' activation function
	- Essentially a 'leaky' atan function
	- Added to init desc.
- Added 'sqr' ativation function
	- Forces a node to be either 1 or -1
	- Added to init desc.
- Renamed .Train() -> .ExpTrain()
- Renamed .FastTrain() -> .Fit()
- Increased default Ieta to 9 in .Fit()
- Increased default gamma to 3 in .Fit()
- Removed 'sz' variable at end of fastCalc(), replaced directly with inVector.size


V1.1.1
- Added ScoreFunction to batch section of FastTrain
- Fixed some typos
- Adjusted the progress bar in FastTrain


V1.1.0
- Added "RND" a.f. (linear but rounds end result)
	- Added description to AdvNet __init__
- Changed legend titles in Forecast()
- Added r2d2() as an easy r^2 function
- Added SSE (renamed AAE) as a method option in netMetrics()
- Changed genTrain() desc to mention prioritizing .FastTrain()
- Added .FastTrain method which is un updated/simplifed genTrain
- Added dataSelect() as a data thinning method
	- Uses 2nd derivative ("curvature") of points for making weights
	- Only useable for 1D X and Y arrays
- Deprecated thinData()


V1.0.0 (True Machine Learning!)
- Finally!(!1!) have a proper training method to fit to basically
  whatever data is thrown at the algorithm
	- Added .Train method for AdvNets; returns a trained version of net
- Renamed first training function Train() to OldTrain()
- Added TTSplit() (train-test split)


V0.3.0 (AFs and Calculate stuff)
- Renamed self.calcTime to self.speed
- Added RESU activation function (has R2 ~= 0.9 for y = x^0.5, x^2 & sin(x))
	- Currently experimental
	- Using both (-) and (+) ends currently
		- Unlike RELU which is just (+) end
- Added RESU2
	- Similar peice-wise behavior to RELU, just with RESU calc instead
- Added RESU and RESU2 to __init__ description
- Renamed "NONE" to "LIN"
- Added lower-case/other handling for __init__ and applyActi() for activation function name
- Added useFast option to genTrain, netMetrics and forecast


V0.2.8 (Misc. Features)
- Moved activation function application to a function applyActi()
	- .Calculate now uses this
- Added calcTime as a net property
	- Is printed out along with other __str__ stuffs
	- iterations is 2 (I=2) for this calculation
- Updated .Calculate() inVector shape/misc. checking
	- Removed try/except around the rowtest in .Calculate
		> this shouldn't be needed as a vector type check happens just before
	- Removed try/except around checking for ndarray type (replaced with type() if statement)
	- Added ValueError(f"Expected inVector of size {(self.inSize, 1)} or {(1, self.inSize)}, got {(np.size(inVector, axis=0), np.size(inVector, axis=1))})")
	> Could just do try/except when doing matrix multiplication, but current way give more info for errors
	- Now checks for correct vector shape explicitly
- Added fastCalc
	- Takes 40-70% of the time compared to .Calculate
	- Skips most handling checks in favor of speed
	- Still supports single float inputs/outputs
- Added fastCalc as option to .Calculate()



V0.2.7 (Hotfix)
- File loading error
	- Happened when creating a "linear regression" net (ie hiddenSize = [])
	- FIX: Added "ndmin=1" to np.loadtxt() for loading the activations
		- This should not be neccessary to add for the loading of
		  the size of weights, as sizes will always have at least
		  2 entries, and the loaded weights are not iterated over


V0.2.6 (Hotfix)
- Still encountering file errors
	- Removed the "./" infront of the paths when saving/loading
	- Commented out "name = str(name)" in LoadNet and SaveNN
	- These didn't fix it


V0.2.5 (Stupidity Fix)
- Deleted old_versions from local area
	- Was creating an exponential file size increase whoops
	- Just retrieve old versions from PyPI if for some reason needed


V0.2.4 (Hotfix)
- Fixed loading nets weights from DIR, not code's folder


V0.2.3
- Added optional weight change return in TweakWeights() for usage with gradient decent stuff later
- Nets (finally) save and load from their own folder, located along with the code
- Added ApplyTweak() to easily make use of what is (optionally) returned from TweakWeights in training
- .Calculate Vector fixes:
	- Added "raise ValueError(f"Net input size of 1 expected, got {self.inSize} instead. It's also possible the inVector is simply not a numpy array/vector")"
	- Moved handling for row vectors to before actual calculation stuffs
	- Before first calculation, added another "calcVec = calcVec.reshape(calcVec.size, 1)"


V0.2.2
- Updated ATAN to the numpy function (faster now lol)
- Updated ELU to be a true piece-wise exponential and linear function (not the weird thing it was before)
- Moved the old ELU definition to "EXP"
- Added the SIG (sigmoid) activation function
- Remove outdated .Calculate() string info
- Updated __init__ info string for SIG fucntion
- Updated activation function list check for too many OR too little functions provided
- Removed vector reshaping for yHat in netMetrics -> SSreg


V0.2.1
- Added >1D handling for yData in thinData()
	- Updated description
- Added 'gamma' as decay factor in genTrain()
- NOTE: found it is better to increase batch size (essentially search depth) when training
		rather than increasing iterations (and/or decreasing gamma)
- Added 'smart' batch sizing to genTrain()
	- Default is 0 which calls a depth/batchSize from 20 to 10
	- This depth value exponentially decays from iteration 1 to ~1000
- Fixed end print statments printing with silent mode on in genTrain()


V0.2.0
- Fixed 'silent' printing in CycleTrain
- Changed Forecast legend to "validation data" for comparison/Y data
- Changed forecast vali data plot to "--" type
- Added Extend()
	> Increases a net's hidden layer sizes by given int amount
	> Useful for creating large nets to train by "building up" to a larger net 
		-> (normal method) 107.7 s for 8.434 -> 1.983e-8 error
		-> (extend method) 40.2 s for 8.318 -> 3.989e-10 error
	> Imputing zeros (median/random imputing didn't work)
- Added RELU activation type
- Removed Dig() function
- Added net property activationFunction (list of functions to use for each weight layer)
	- Updated Calculate()
	- Updated CopyNet()
	- Updated SaveNet()
	- Updated LoadNet()
	- Removed hiddenFunction input from:
	  Forecast(), netMetrics(), genTrain(), Calculate()
- Added netMetrics()
	> Returns either R^2 or R^2 and the three errors used to find it
- Added genTrain()
	> New training method using a batch of new nets to test each iteration
- Added thinData()
	> Returns:
		>> thinned x points (xThin)
		>> thinned y points	(yThin)
		>> Data point indicies (xPlot)
			>>> Useful for plotting the yThin data, particularly on top of the full yData
	> NOTE:
		>> Using xThin = [*range(len(yThin))] works great for small nets, but technically
		   doesn't train to the true data. Larger nets can train to the true indicies (x data)
		   pretty well which is good, just takes more time (see genTrain #4 pic).
- Depreceated Train() and CycleTrain()


V0.1.5.3
- Changed validationVals to account for 1D vectors (LN588)
- Changed 'default' selection in TweakWeight to 'all'
- Corrected RELU and ELU names (to ELU and ATAN)


V0.1.5
- For nets with inSize = 1, a simple number can be given (no longer required to be that stupid 1x1 numpy array thing).
- Reduced maxCycles default value to 5 for the cycle training


V0.1.4
- Operational as package
- Printing an AdvNet now also prints the # of parameters it has