2

I'm struggling to write my algorithm in a concise and correct way. The following is an explanation for an optimizer's update step of part of a vector of weights (not a matrix in my case).

I have a vector $\alpha \in \mathbb{R}^d$, and a set $S$ that includes some indices $1\leq i \leq d$ ($S \subseteq \{1,\dots, d\}$). Now, I want to denote that $\alpha$ is 0 for every index $i\in S$, and otherwise it's the value as in $\alpha_i$. At first I denoted it $\alpha_S$, but I'm not sure it is properly defined or understandable.

I could use the following notation:

$\alpha_S = \begin{cases} \alpha_j & j \in S\\ 0 & j \notin S \end{cases}$

But its line height is twice the size, and I want to avoid that.

Is there any other formal, simplistic way to notate this correctly? Maybe some kind of a masking vector to be multiplied with $\alpha$?

Thanks!

leed
  • 145
  • 4

2 Answers2

2

Check out the indicator function $1_S(\cdot)$. In your case it would be fined as

$$1_S: \{1, \ldots, d\} \rightarrow \{0, 1\}, j \mapsto \begin{cases} 1 & j \in S \\ 0 & \, \text{else} \end{cases}.$$

Multiplying this function with the respective values should give you what you are looking for.

Edit: If I understand your comment correctly, the vector you are looking for is

$$ \alpha_s = \sum\limits_{i = 1}^{d} \alpha_i e_i 1_{S^C}(i). $$

Here $e_i$ denotes the i-th unit vector of $\mathbb{R}^d$, $S^C$ is the complement $\{1, \ldots, d\} \setminus S$ in $\{1, \ldots, d\}$ and $1_{S^C}$ is the indicator function that is $1$ for $i \notin S$ and $0$ for $i \in S$.

NiklasvMoers
  • 309
  • 1
  • 7
  • So if I have a vector $\alpha$ and I want to "mask it", I would have to do something like: $\alpha \leftarrow (\alpha_1 * 1_S(\alpha_1), ... , \alpha_d * 1_S(\alpha_d))$? Isn't there a vectorized variant of this function to make this simple, like $\alpha \leftarrow \alpha * 1_S$? – leed May 29 '20 at 16:44
  • I meant $\alpha \leftarrow 1_S(\alpha)$ – leed May 29 '20 at 16:50
  • [Indicator vector](https://en.wikipedia.org/wiki/Indicator_vector) does not quite work in my case. – leed May 29 '20 at 16:53
  • This was going to be my first suggestion too: define the indicator vector, and define a coordinate-wise product of vectors. – Ben Reiniger May 29 '20 at 17:59
  • @leed see my edit. – NiklasvMoers May 30 '20 at 11:38
  • You defined a sum over the vector's components. I meant to produce a masked vector. A simple solution to this would be in a for loop: for all $i \in S, x_i = x_i + \alpha_i$. – leed May 30 '20 at 15:11
  • 1
    The vector $\alpha_S$ that I defined is masked as in it has the component $\alpha_i$ in the i-th entry if $i \notin S$ and $0$ if $i \in S$. Notice how I am not adding up the elements $a_i$ but I am multiplying them with the unit vector $e_i$. Therefore, $\alpha_S$ is a d-dimensional vector as well. – NiklasvMoers May 30 '20 at 16:38
0

I like the indicator-and-coordinatewise-product version better, but:

Another option is more geometric: $\alpha_S$ is the projection of $\alpha$ onto the subspace where the coordinates not in $S$ are zero. Perhaps for $S\subseteq [n]$, the notation $\mathbb{R}^S=\{x\in\mathbb{R}^n : x_i=0 \text{ for $i\notin S$}\}$ makes sense? (It's an abuse, for sure, since $n$ is not explicit in the notation, but depending on your needs it may suffice.) Then the desired object is

$$ \alpha_S := \operatorname{proj}_{\mathbb{R}^S} \alpha .$$

Ben Reiniger
  • 11,094
  • 3
  • 16
  • 53