add

Add single key-value pair to KeyValueStore

collapse all in page

Syntax

add(KVStore,key,value)

Description

add(KVStore,key,value) adds a single key-value pair to KVStore, which is a KeyValueStore created during mapreduce execution. Use add in a map or reduce function written for use with mapreduce to store intermediate or final key-value pair information.

example

Examples

collapse all

Open Live Script

Use add in map and reduce functions to pass data into the intermediate and final KeyValueStore. This example uses identity map and reduce functions that pass the inputs straight through to the output. The map and reduce functions are listed at the end of the example as local functions.

inds = tabularTextDatastore('airlinesmall.csv','SelectedVariableNames','ArrDelay','TreatAsMissing','NA');
preview(inds)
ans=8×1 table
    ArrDelay
    ________

        8   
        8   
       21   
       13   
        4   
       59   
        3   
       11   


outds = mapreduce(inds,@myMapper,@myReducer,mapreducer(0));
********************************
*      MAPREDUCE PROGRESS      *
********************************
Map   0% Reduce   0%
Map  16% Reduce   0%
Map  32% Reduce   0%
Map  48% Reduce   0%
Map  65% Reduce   0%
Map  81% Reduce   0%
Map  97% Reduce   0%
Map 100% Reduce   0%
Map 100% Reduce 100%

readall(outds)
ans=1×2 table
        Key               Value      
    ____________    _________________

    {'ArrDelay'}    {123523×1 double}

Local Functions

function myMapper(data,info,intermKV)
    add(intermKV, 'ArrDelay',data.ArrDelay);
end

function myReducer(key,intermValIter,outKV)
    data = getnext(intermValIter);
    while hasnext(intermValIter)
        data = [data; getnext(intermValIter)];
    end
    add(outKV,key,data);
end

Input Arguments

collapse all

Key-value pair storage object, specified as a KeyValueStore object. The mapreduce function automatically creates the KeyValueStore object during execution:

  • In the map function, the name of the intermediate KeyValueStore object is the third input argument to the map function, myMapper(data, info, intermKVStore). Use that same variable name to add intermediate key-value pairs with add or addmulti in the map function.

  • In the reduce function, the name of the final KeyValueStore object is the third input argument to the reduce function, myReducer(intermKey, intermValIter, outKVStore). Use that same variable name to add final key-value pairs with add or addmulti in the reduce function.

For more information, see KeyValueStore.

Key, specified as a numeric scalar, character vector, or string.

All of the keys added by the map function must have the same class. The keys added by the reduce function also must have the same class, but that class can differ from the class of the keys added by the map function.

Numeric keys cannot be NaN, complex, logical, or sparse.

Example: add(intermKVStore,'Sum',sum(X)) adds a key-value pair to an intermediate KeyValueStore object (named intermKVStore) in a map function.

Example: add(outKVStore,'Stats',[mean(X) max(X) min(X) var(X) std(X)]) adds a key-value pair to a final KeyValueStore object (named outKVStore) in a reduce function.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | char | string

Value, specified as any MATLAB object. This includes all valid MATLAB data types.

The OutputType argument of mapreduce affects the type of values that the reduce function can add:

  • If the OutputType is 'Binary' (the default), then a value added by the reduce function can be any MATLAB object.

  • If the OutputType is 'TabularText', then a value added by the reduce function can be a numeric scalar, character vector, or string scalar when using the add function. Additionally, you can use the addmulti function to add multiple values with a numeric vector, cell vector of character vectors, cell vector of numeric scalars, or string array. In each case, the numeric values cannot be NaN, complex, logical, or sparse.

Note

The above key-value pair requirements may differ when using other products with mapreduce. See the documentation for the appropriate product to get product-specific key-value pair requirements.

Example: add(intermKVStore,'Sum',sum(X)) specifies a single scalar value to pair with a key.

Example: add(outKVStore,'Stats',[mean(X) max(X) min(X) var(X) std(X)]) specifies a numeric array as the value to pair with a key.

Tips

  • Avoid using add in a loop, as it can negatively affect mapreduce execution time. Instead, use cell arrays to collect multiple values (using vectorized operations if possible) and use a single call to addmulti.

Version History

Introduced in R2014b

See Also

|

Topics