| Author |
Message |
FreeSynapse
Joined: 11 Jun 2006 Posts: 39
02-03-07, 01:24 pm |
Post subject: Avoiding parameter hell |
|
|
| If a parameter can correctly be made auto-tunable, it should be! Users shouldn't have to go through optimizing a half dozen parameters to arrive at a good model. For example, already with ANNs, I can think of so many parameters that need to be optimized, including: (1) the choice of training algorithm, e.g. backpropogation (2) the number of layers (3) the type of each layer (4) the number of neurons in each layer (5) learning rate (6) momentum. It's workable, but so not ideal. This doesn't even include additional parameter optimization for preprocessing and ensembles. |
|
| Back to top |
|
| Author |
Message |
intelligent robot
Joined: 03 Sep 2006 Posts: 35
02-17-07, 06:35 am |
Post subject: |
|
|
The parameters can't be made auto-tunable without long rounds of testing.
You must remember that the HTM in our heads was 'tuned' with millions of years of evolution. Tuning an HTM will require similar amounts of trial-and-error. |
|
| Back to top |
|
| Author |
Message |
FreeSynapse
Joined: 11 Jun 2006 Posts: 39
02-19-07, 11:52 am |
Post subject: Dynamic adjustment |
|
|
| At least there are certain parameters that can be dynamically adjusted during the training process, like the learning rate and the momentum, etc. in an ANN. |
|
| Back to top |
|
| Author |
Message |
chatham
Joined: 25 Mar 2005 Posts: 64
03-05-07, 08:38 am |
Post subject: bad idea |
|
|
Technically, this would definitely make modeling easier.
But this is a theoretically unsound thing to do.
Any model can fit any data given enough free parameters. The most crucial aspect of cognitive modeling is biological plausibility - in other words, you should know what parameters will work in your model based on the biological analogues of those parameters.
For example, you can get a good idea of what the learning rate should be depending on whether you're talking about hippocampus (fast), sensory cortex (slow) or prefrontal cortex (very slow). Likewise, the size of your layers can be determined based on the functionality you need from the layer, as well as the relative size of the brain regions that layer is meant to represent.
Many modelers ignore these biological constraints, and make more abstract "information processing" models of cognition, but they do so at the expense of knowing whether their model is actually accurate, as opposed to merely fitting behavioral data via careful setting of free parameters. _________________ http://scienceblogs.com/developingintelligence |
|
| Back to top |
|
 |
Page 1 of 1 |
All times are GMT - 8 Hours
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2002 phpBB Group
Please contact the board administrators if you have any questions regarding the OnIntelligence.org forums.
|
| |