realtime data streaming between matlab and opencv, visual c++

My C program tracks any particular colour in a video and prints the coordinates of that point. I want to write a matlab program which can take these coordinates as input and analyse them.I am doing all this in windows XP. I have read about mex files and also about the memory mapping but, nothing is working.

When I build my mex file, it shows many errors with the core.hpp file like:

"ptrdiff does not belong to the class std"

can anyone help?

Answer 1

[Geoff]

How "realtime" do you need? If you were polling a file multiple times a second, would that be sufficient? Can you do your analysis in blocks, or must it be live? After analysis in MatLab, do you need to deliver a result back to the C++ program? How complicated is your analysis - ie is there any reason not to do it in C++?

The best option in my opinion would be to try and get UDP communication going. Have your tracking application simply fire off the coordinates to localhost using UDP on some port. If your MatLab script is listening, it will start receiving the data and can do something with it. If it needs to communicate back, have it fire off UDP packets on a different port, which your tracker is listening on.

Presumably you are comfortable with threading, synchronisation and network communications?

A quick Google search pulled up something vaguely promising:

http://webscripts.softpedia.com/script/Scientific-Engineering-Ruby/TCP-UDP-IP-Toolbox-36497.html

Or there is TCP/IP code if you feel like bashing your head against a wall:

http://www.mathworks.com/matlabcentral/fileexchange/21131

But if polling a little file is responsive enough, do that. I would just use binary so I could be sure I had read the entire co-ordinate. I'd know how many bytes to expect and, if I appended to the file (preferable to avoid syncronisation issues or data corruption) it would contain a complete record for post-analysis if necessary.

[Geoff]

Please don't email me with your comments. Then I am the only one with information and nobody else can help you.

You wrote: The analysis must be live. I am a newbie so am not familiar with threading, synchronisationand network communications. The reason for not doing this in C++ is that my main program is written in matlab and rewriting it in c++ is a pain as I am a beginner in the field of programming and i dont have much time. I have only week left to finish this project.

[Geoff]

You wrote: I do not need to send back any message to c++.

I used the C++ program to track the coordinates of a moving object as matlab gets slow. so communicating the output with matlab and giving the final output should be at at the speed of around 30 frames per second of the video input.

I went through that google link before posting this question but was not sure of it.and i never used UDP. but i shall try it.any other simple ways to get around this problem?

[Geoff]

So I suggest you set up a timer to fire every 2 to 10 milliseconds or thereabouts. A quick test showed me I could open a file in binary mode, read 16 bytes and close it about 11,000 times a second so that's fine. You could of course just keep the file open and seek to the correct place. C++ app can just append to it so it becomes a stream. You would open it, seek to the end (aligned to whatever your data packet size is), then set a timer to read a co-ordinate. If you are able to read a whole co-ordinate, then process it - otherwise seek backwards by the number of bytes you just read and wait until the next timer hit. What is the datatype of the co-ordinates you are writing, and how many values do you output for each frame?

[newbie]

the x and y coordinates. data type is double.

[Walter Roberson]

rmmd: if you are not familiar with threading, synchronization, and network communications, you are not likely going to be able to complete your project within one week using those facilities, not unless you were to hire a consultant familiar with those technologies and with MATLAB. (The consultants I know of who might have the proper qualifications are already each booked up this week.)

Geoff's approach of using a file is more likely to have success within a limited time.

Unless, that is, you happen to be using an MS Windows system, in which case you are likely to run into problems opening the file on one of two sides. In theory those problems can be overcome in MS Windows, but the technical issues involved are tricky and not readily solved except by someone with a fair bit of Operating Systems experience.

[Geoff]

I anticipate the problem would be file buffering on the write side. I would first attempt to write the file using functions from stdio, and try fflush(). If that didn't reliably flush to disk, I'd try something lower-level. This involves opening the file with no caching or buffering. See MSDN for CreateFile(). You have to write a whole sector at once, so you need to be comfortable with data manipulation in C and probably have some insight into data protocols -- This is because your MatLab program can no longer read to the end of the file (since a lot of it will be padded to fill the sector) and you would need to include some basic control signals in the file. Something as simple as a stop-byte to terminate the co-ordinate should be fine... Your packet would look like this: [X_8bytes][Y_8bytes][CTL_1byte]. Therefore 17 bytes per co-ordinate. I would zero-pad the file and make the stop-byte anything non-zero. 0xff would be fine. I don't know how well MatLab would cope with reading this file, as I'm not familiar with its buffering scheme.

[Walter Roberson]

I'm not thinking so much about the caching as about the problem of having a file open in two places. Even if you did open/read/close cycles since the two ends operate at different speeds you risk trying to open while the other process has it open. So you need a mutex implemented somehow, which is tricky to do right via file-system (especially on a networked filesystem.) Semaphore would probably be the most secure... provided the two processes are executing on the same computer.

A bunch of this can be alleviated if you can defeat the automatic locking mechanism that MS Windows uses that prevents a file that is opened for reading from being opened by another process. I don't know how to do that: the existence of that lock is not ISO compliant, and when I read the MSDN documentation about open() options and fctl() and the other standard mechanisms for controlling such matters, I cannot find any reference to that automatic lock existing at all. :(

[Geoff]

Walter, I didn't believe you about having file locking problems, and I expect this is to be run on a local machine and local disk. So I decided to benchmark it. I made a simple source/sink console application using stdio functions, and had it output packets containing the performance counter value at the source. The sink would read these immediately after querying the performance counter and output the packet lag. Source and sink ran as two separate processes. At 30fps, including a 1ms Sleep in the read loop, the lag peaked at 3% of a frame. I ran it at 500fps with no Sleep and the peak was about 7% lag (with the Sleep it was around 60%, which is to be expected). In short: communication via file is perfectly viable. At least, in C. I did not test this using MatLab as the reader, but I don't see why it would be a problem. Just make sure you call fflush() after writing. You can have the file open for write on the source and read on the sink simultaneously with no problems.

[Geoff]

Okay, had some more fun with this. I ported my Sink code to MatLab and confirmed this method works fine - average lag was 0.05% of a frame with very occasional peaks up to about 2%. No frame dropping observed. Caveat is that you need to busy-wait (ie poll the file stupidly fast) or MEX a better pause() function. I would take the simplest route and poll very fast, because this at worst is only going to max out one core. Anyway, that core is going to spend the rest of its time processing co-ordinates, so getting hold of a co-ordinate ASAP means you have about 30ms to process it.