Higher-Order Function

Higher-Order Functions are functions that operate on functions (take functions as at least one of their parameters).

Map

Purpose: Create a new array by applying a function to each of the elements of an existing array. o

Not to be confused with a Hash Map

Reduce

Purpose: Apply a binary function over an array to produce a new final answer. (e.g. multiply all integers in an array)

From CS336: Reduce pieces from all ranks to rank 0, applying some operation (e.g., sum, min, max)

# Input
rank0 = tensor([0.])
rank1 = tensor([1.])
rank2 = tensor([2.])
rank3 = tensor([3.])
# Output
rank0 = tensor([6.])  # Sum of all ranks (0 + 1 + 2 + 3)