Why is my array so big?

Hello,

I am trying to make a 1 dimensional vertical array of about 5500 strings. Even better, maybe convert the whole thing to a single string using Char()??

Anyway, I need to concatenate several "strings" together, and then store in one array location. But it seems it's not counting the concatenated string as one string, because I got the message the array was too large to be displayed, which shouldn't happen.

The code is too convoluted to post it all, but I believe the relevant pieces are below:

I initialize like this-

AoA = repmat({''}, 5589, 1);

I build up my string in a couple steps. I have a loop that tacks on pieces to this string-

AoAr = [AoAr ',' num2str(mean(AoA_sim(ind)))];

...so I hope that winds up being one long string. Then I tack that onto this-

AoA{k,1} = strcat(num2str(set_theta), ',', ...
           num2str(Wr_r_grid(k)*S{NR+2,1}/v_r_grid(k)), ',', AoAr);

...where I again hope the strcat and num2str's force all that to really be just one string that takes up one cell.

Finally I do a-

AoAout=char(AoA);

...and when I try to view AoAout, it says "Cannot display summaries of variables with more than 524288 elements.". So I assume I'm not handling the data properly.

I think it's not smooshing down like I had hoped. I probably went wrong in several areas...can someone help me out?

I appreciate it in advance!

Dave

[dpb]

I'd suggest quit relying on "hope that winds up being one long string" and check what your code actually does and see where it is/is not what you expect.

num2str will return a string that is a row vector (array) of character; I can't decipher what you really mean by the "1D vertical array". Do you really mean a 1xN character array or a cell array of length N, each cell of which is a cell string?

Give a small sample of the input and a desired output by way of clarification...only 3 to 5 elements should be sufficient to illustrate; 5000+ is simply larger of the same...

ADDENDUM

BTW, it doesn't say the array isn't existant or some such only that "Cannot display ...". IOW, you've created a monster-sized array and the display i/o is smart enough to not to try to spend 20 minutes scrolling a screen.

What does

whos AoAout

return?

As a guesstimate, say each numeric conversion from num2str averages as many as 15-16 characters or so, then the ~6000 elements would still be <100k characters.

[David Pesetsky]

Sorry for the lack of detail...I'm just starting to code.

I'm shooting for something I can copy into excel, and delimit by commas. So, the output "array" should be 5589 vertical elements, where each element is a string I put together with the values and delimiters I had in mind. So like:

-4, 2.83, 89.65, 81.78, ... -4, 4.23, 81.60, 78.18, ... . . .

The inputs come from me trying to join num2str results and commas using strcat.

The whos AoAout command gives:

AoAout 5589x382 4269996 char

I agree the output should have 5589 rows, but I'd like it to have 1 column by actually joining the string pieces together into one element.

Hope that makes more sense....

Dave

[Star Strider]

Why use num2str at all?

I would use csvwrite, xlswrite, or other appropriate function.

What version of MATLAB do you have, and what spreadsheet read/write functions do you have available?

[dpb]

Indeed, as Star says, you don't need the conversion at all for this purpose; for numeric values it'll be better for almost any purpose to have them as numeric in the spreadsheet as well rather than character.

If you only need a csv file, then csvwrite is the tool again rather than converting first.

It's sometimes overwhelming w/ Matlab owing to the large number of functions to know who to choose initially; an overlooked tool is help

Start w/ it alone and then look at the subtopics...in this case it's pretty easy to see that input/output is the area of interest so follow up with

help iofun

Under that look at the list for input/export and choose those that look interesting/suitable...

I'll also note, however, that in Matlab it's generally better to have the full array of data and then use the "vectorized" form a a function rather than the dynamic building of arrays with loops and concatenation as you're doing.

If you really did want to build a character array of the values of a group of variables, then

>> dat=[-4, 2.83, 89.65, 81.78,  -4, 4.23, 81.60, 78.18];
>> num2str(dat,'%f,')
ans =
-4.000000, 2.830000,89.650000,81.780000,-4.000000, 4.230000,81.600000,78.180000,
>> num2str(dat.','%f,')
ans =
-4.000000,
2.830000,
89.650000,
81.780000,
-4.000000,
4.230000,
81.600000,
78.180000,
>>

The difference you'll note is the .' element-wise transpose operation to orient the input by column instead of by row and then num2str is "smart enough" to follow the lead in it's operation.

But, there's little reason to do this as a general practice; it's suitable when generating labels for labeling graphs or other such purposes but rarely to actually build output for input/output; there are better tools for that specifically for the purpose.

[David Pesetsky]

I am using Matlab v2012a.

I tested xlswrite('AoA.xls',AoA); on a smaller AoA array, and it does populate excel with what I'm looking for, but I use it after the AoA array is completely full. I'm concerned about taking up unnecessary array space with incorrect coding and possibly hitting a limit that I shouldn't hit because it thinks I have 382 columns instead of 1.

[Star Strider]

Experiment. Your fears may be unfounded.

That’s the only way you’ll ever know for sure.

[David Pesetsky]

Hi dpb, I think my formatting gets messed up when I post. What I want to have is:

-4, 2.83, 89.65, 81.78
-4, 4.23, 81.60, 78.18

So I would get several lines of text, but each line is really one string. I think if I sneak a \n in your dat example, I may get that.

I want to both store the array in the matlab structure, with the option of exporting to csv. That's why I want to store it "efficiently", in case it gets large.

[dpb]

To preserve line wrap, separate the section with a blank line and select then use the Code button or insert two blanks in front of the first line. It's irritating but until TMW finally figures out to turn word-wrap off it's the only way.

Anyway, to store efficiently, keep the data as numeric. As character

>> s=['-4, 2.83, 89.65, 81.78'];
>> whos s
Name      Size            Bytes  Class    Attributes
s         1x22               44  char               
>>

The same values as default double are

>> v=str2num(s);
>> whos v
Name      Size            Bytes  Class     Attributes
v          1x4                32  double              
>>

So, even for that small sample it's an overhead of 12 bytes extra to store as a string or a percentage of 44/32-->11/8 = 37.5% increase in storage.

Again, when there's a need for i/o, then convert, otherwise you're just wasting space (and accuracy) by converting to a character representation.

[David Pesetsky]

OK, I think you convinced me for numerical accuracy at least to use numbers.

I thought there must be a way to join the mini-strings in your S so that whos would register it as 1x1.

[dpb]

Well, there is, but you have to look at "the rest of the story"

>> c={s}
c = 
  '-4, 2.83, 89.65, 81.78'
>> whos c
Name      Size            Bytes  Class    Attributes

c         1x1               104  cell

>>

c is a one-cell cell array but it now takes 104 bytes to store the cell string of 22 characters (44 bytes) or 4 doubles (32 bytes). That's even more overhead for the luxury of the cell array.

Cell arrays and cell strings have their place (they're the only way to store "jagged" arrays, for example) but they and the other higher-level datatype abstractions come at a price. "There is no free lunch"

Answer 1

[David Pesetsky]

To close this issue, here's what I've implemented.

I decided to use the "more expensive" cell array, since I have strings mixed with numbers, but it is rectangular.

Initialize-

AoA = zeros( 5590, 50 );

I load up the array with a nested for loop-

AoA{k+1,j} = mean(AoA_sim(ind));

The step that was complaining about "too big to display" I think was doing-

Char(AoA);

...which really isn't needed.

It also exports nicely to xls-

xlswrite('AoA.xls',AoA);

Thanks all, Dave

[dpb]

"...to use the "more expensive" cell array, since I have strings mixed with numbers, ..."

That's another reason for cell arrays, yes...