Forum Home | User Profile | Register | Members | Groups | Search | FAQ | Back to:OnIntelligence.org
onintelligence.org Forum Index -> Models and Simulation Topics -> Avoiding parameter hell

 
This topic is locked: you cannot edit posts or make replies.   This forum is locked: you cannot post, reply to, or edit topics. View previous topic :: View next topic  


Author Message
FreeSynapse


Joined: 11 Jun 2006
Posts: 39

02-03-07, 01:24 pm
PostPost subject: Avoiding parameter hell Reply with quote

If a parameter can correctly be made auto-tunable, it should be! Users shouldn't have to go through optimizing a half dozen parameters to arrive at a good model. For example, already with ANNs, I can think of so many parameters that need to be optimized, including: (1) the choice of training algorithm, e.g. backpropogation (2) the number of layers (3) the type of each layer (4) the number of neurons in each layer (5) learning rate (6) momentum. It's workable, but so not ideal. This doesn't even include additional parameter optimization for preprocessing and ensembles.
Back to top
View user's profile Send private message

Author Message
intelligent robot


Joined: 03 Sep 2006
Posts: 35

02-17-07, 06:35 am
PostPost subject: Reply with quote

The parameters can't be made auto-tunable without long rounds of testing.

You must remember that the HTM in our heads was 'tuned' with millions of years of evolution. Tuning an HTM will require similar amounts of trial-and-error.
Back to top
View user's profile Send private message

Author Message
FreeSynapse


Joined: 11 Jun 2006
Posts: 39

02-19-07, 11:52 am
PostPost subject: Dynamic adjustment Reply with quote

At least there are certain parameters that can be dynamically adjusted during the training process, like the learning rate and the momentum, etc. in an ANN.
Back to top
View user's profile Send private message

Author Message
chatham


Joined: 25 Mar 2005
Posts: 64

03-05-07, 08:38 am
PostPost subject: bad idea Reply with quote

Technically, this would definitely make modeling easier.

But this is a theoretically unsound thing to do.

Any model can fit any data given enough free parameters. The most crucial aspect of cognitive modeling is biological plausibility - in other words, you should know what parameters will work in your model based on the biological analogues of those parameters.

For example, you can get a good idea of what the learning rate should be depending on whether you're talking about hippocampus (fast), sensory cortex (slow) or prefrontal cortex (very slow). Likewise, the size of your layers can be determined based on the functionality you need from the layer, as well as the relative size of the brain regions that layer is meant to represent.

Many modelers ignore these biological constraints, and make more abstract "information processing" models of cognition, but they do so at the expense of knowing whether their model is actually accurate, as opposed to merely fitting behavioral data via careful setting of free parameters.
_________________
http://scienceblogs.com/developingintelligence
Back to top
View user's profile Send private message Visit poster's website

Display posts from previous:   
This topic is locked: you cannot edit posts or make replies.   This forum is locked: you cannot post, reply to, or edit topics.    Page 1 of 1 All times are GMT - 8 Hours

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group

Please contact the board administrators if you have any questions regarding the OnIntelligence.org forums.