[Elite]

Hey all!

Has anyone actually run Dileep George's Matlab code not just demonstration, but also for the actual training? I have read through all topics in this forum but did not find anybody who experimented with training the model.
Well, I made it to learn from scratch, and it is definitely not a one step process! (Took nearly two weeks to figure everything out). So I decided to give some guidance to someone who may also want to experiment with the code. This will serve as a preliminary documentation, which could not be found anywhere before.

1. Download all the training files from http://www.stanford.edu/~dil/InvariancePaper/ImagesJpg/ (will need something like TeleportPro to automate this). Select just 32x32 images, discard all the rest.
2. Convert all resulting jpg images to bitmaps of 1-bit depth.
3. Make a directory structure for inputting to the program. There are two images for each category. Each category must have its own folder with its images in it. So we end up with 91 folders with 2 images each. (Folder "Bed" contains bed.bmp and bed2.bmp, and so on).
4. Download and unzip version 1 code from http://www.stanford.edu/~dil/invariance/Download/InvarRecogRel1.zip and Pictionary Demo code from http://www.stanford.edu/~dil/invariance/Download/PictionaryDemo.zip. The latter is the basis project which we will work with.
5. Modify learning_main.m code to read files properly (if running on Windows). Basically, the code needs to use '\' for path separators and ignore files such as "thumbs.db".
6. Do the same changes in trg_categories_and_examples.m.
7. Rename function and file initialize_tree_for_inference_no_overlap to InitTreeForInferenceNoOverlap as the name is too long.
8. Copy files Level1_group_constru.m and Level1_vectors.m from Version1 code to Pictionary code (they are missing from the latter!).
9. Comment out "grid minor" from sketch4.m (Matlab 6 does not compile this).

After this preparation, the steps to train the code are as follows:

clear;
Level1_vectors;
[LearnedTree, L2_vectors] = learning_main('..\\Training images', 'construct_tree_noovlap');
TreeForInference = InitTreeForInferenceNoOverlap(LearnedTree);
TrainingCategories = trg_categories_and_examples('..\\Training images');

Here "Training images" is the folder containing 91 category subfolders. After this code completes (learning takes 24 hours (!) on Athlon XP 2000+, 512 MB), save the variables L2_vectorsNoOvlap, TrainingCategories and TreeForInference to the files with the same names in the Data folder. Your Data folder must also contain UniqueL1_vectors.mat (I didin't find yet where this is generated, but this is a rather simple matrix).

Now clear the workspace, run the demo_wrapper and it opens with the 91 categories, ready to recognize any of them!

I will let you know about future experimentation with the code. Best regards.

[magdeburg]

Hallo all,

Could everyone tell me about the estimation/recognition quality of this demo? I used the original version and inputed a new image (I painted) for the letter "A" for example, but the system can not recognize it as well. Did anyone modify this code for loading a training image and test if it can be also recognized exactly?

Thanks.

[Elite]

[magdeburg]

[momob]

Hello guys,

few weeks ago I went to Elite web site not expecting much new stuff. Boy I was wrong!!! Not only it has papers that he wrote which are 100% easier to read than I found so far on the memory-prediction concept but the software test-bench is incredible and so easy to use. For instance I set a quick experiment with 5 objects (square, rectangle, face, plane and triangle). I hand draw all of them and had the system learned them (few seconds). Then I simply draw a a figure like a face (does not even look like the faces I draw in the training set) but still the system figure out what it is. I even draw half a face and the system did figured out (know I know why you can make out an object half hidden behind a wall for instance!). I added some noise pixels just for fun. No problem had no problem. I then draw a part of triangle. Once again the system guessed i right!

I am simply amazed about the accuracy and the noise tolerance of Elite system.

So if you are serious learning about "computer learning", check out Elite site!! You won't regret it (I sure don't, I downloaded all his papers and read them on flight to Phoenix....fun read).

Thank you so much Elite for charing your research with us. Good luck for your future experiments.

Mo.

[Elite]

Wow Thanks for writing this review and sharing your impressions, Mo!
_________________
My research page on memory-prediction framework: http://www.phillylac.org/prediction/
Open-source version of the framework: http://sourceforge.net/projects/neocortex/

[magdeburg]

[momob]

Hello magdeburg.

Are you drawing a square box (the sign itself) and then drawing the speed limit inside the box like 50,25...miles)? It maybe difficult for the system to reconignize the text inside the box. All I can tell you that I was very impress with the system so far. For instance I "teach" it faces (also triangle, planes,squares ...) and then for fun I draw a small square, then another one close by. The system said it was looking at a square. So far so good. The I draw a vertical line (a noze) between the two squares (eyes!) it told me it was looking at a face!!!!!!!!!!!!!!! adding the shape of the face (a simple circle around the eyes and noze) the system was even more confident that it was looking at a face. I found this uncanny!

In any event, it maybe that because the limited resolution (32x32 pixels), the system has difficulty to reconnize small images. I just cannot wait when the system (eventually I am sure) would be capable of higher image resolution (256x256???). Then you would really be able to give it a much more sofisticated images to recognize.


Just my 0.02 of course!


Mo.

ps: To the forum admin: It will be helpful to be able to upload graphic files to show what people are using for training/test images. it will be easier to share info among us. Thanks.

I

[Elite]

In fact, Magdeburg is experimenting with a photo-like pictures of European traffic signs - a circle having a speed limit number in it. This has little to do with the resolution of the picture being recognized. The problem with the pictures (or rather with the initial approach of modeling memory-prediction framework) is that these training images consist of THICK lines (2-4 pixels thick). There is a thick circle and a "bold" number inside of it.

As I wrote in my latest research paper, the initial model is supposed to be trained on pictures that are combinations of straight vertical/horizontal lines, right corners and line intersections (all lines are 1 pixel in width). Why? Simply because these primitives are hardcoded into the lowest level subregions. So, for example, when this 4x4 pixel subregion sees a part of a thick line, it may even see all 4x4 pixels black! And since it 'knows' only the above primitives, it must classify this pattern as one of the known ones. But this is too different from anything the subregion knows to be classified correctly. So it is no wonder that this learning fails. Of course, this is a major drawback of the initial MPF model.

I addressed this drawback by making the bottom level subregions capable of LEARNING their patterns, instead of relying on the predefined set. This is demonstrated by the "MPF With Forgetting.exe" program that can be downloaded from my website. I tried using this with Magdeburg's training data and the recognition performance was quite good. I don't know if Magdeburg tried it with this program, or just used the initial version. It would also be helpful to see what testing data he used.

Saulius
_________________
My research page on memory-prediction framework: http://www.phillylac.org/prediction/
Open-source version of the framework: http://sourceforge.net/projects/neocortex/

[magdeburg]

[Elite]

[gregko]

Saulius,
I read your excellent article and played briefly with your code, and also read Numenta articles. It seems to me that George/Hawkins in their article and Matlab code (and also the Numenta white paper) emphesizes the learning of common sequences of patterns, something with I did not see mentioned in your article. Did you omit this part in your experiments with HTM?

Greg

[Elite]

Greg,

I suppose you are talking about TEMPORAL SEQUENCES of spatial patterns and wondering why I only mention learning of spatial patterns, not sequences.

My software and George's Matlab code uses the same type of 'sequences' while recognizing images. However, what the system really learns is patterns, not temporal sequences. So in this sense their paper was more generalized than their code: the paper spoke about sequences, while the code learnt and memorized only spatial patterns. There is some confusion of what is called sequence in these papers. Hawkins' book describes temporal sequences that are remembered and then retrieved from memory sequentially over time. George's papers and Matlab code, however, mention 'sequences' as showing the same pattern in several different places (input frames) of the input field with a possibility of flipping it. The difference is, these 'sequences' exist only in the input data, they are NOT remembered. What is remembered is a probabilistic representation of possible patterns based on ALL these inputs. Thus there is no time component in memory. If we show the same input frames in any other possible order, this will have no effect to the recognition performance. So in this case the term 'sequence' is confusing, it is more like a SET of frames, not a temporal sequence. This is why I do not speak about sequences in my paper in order not to confuse the reader.

Saulius
_________________
My research page on memory-prediction framework: http://www.phillylac.org/prediction/
Open-source version of the framework: http://sourceforge.net/projects/neocortex/

[gregko]

Thank you, Saulis. You were right not to confuse the matters any more than necessary. I gave this some thought since I posted my question to you, and came to the same conclusion, that what George calls "sequences" are groups of similar patterns, like all possible combinations of a bottom left corner that a node looking at 4x4 pixel patch would see.

Apparently this type of grouping and reporting up the hierarchy the "group" or "set" number has advantages in reducing numerical complexity and memory usage. But how to make the nodes learn such meaningful grouping automatically, on all the levels? We could do this at the bottom level, as we can easily imagine 4x4 pixel patches and what would be meaningful here, but HTM nodes are supposed to learn by themselves what is meaningful and how to group them.

Greg

[dileep]