Google

A parameterized function can be expressed generally as $$o = g(x_0, x_1, \dots, x_m; \theta)$$. In this formulation, $$o$$ is the output, $$g$$ is the function, $$\left\{x_0, x_1, \dots, x_m\right\}$$ is the set of input variables, and $$\theta$$ represents the set of parameters that define the function's specific behavior. The parameters are typically separated from the inputs by a semicolon.

General Formula for a Parameterized Function

A parameterized function can be represented in two equivalent ways. The general form, $$o = g(x_0, x_1, \dots, x_m; \theta)$$, explicitly lists the parameters $$\theta$$ after the input variables. A more concise shorthand notation, $$g_\theta(x_0, x_1, \dots, x_m)$$, moves the parameters to a subscript of the function name. The equivalence is expressed by the equation: $$o = g(x_0, x_1, \dots, x_m; \theta) = g_\theta(x_0, x_1, \dots, x_m)$$. This subscript notation emphasizes that the function's behavior is determined by the parameter set $$\theta$$.

Equivalence of Parameterized Function Notations

Diverse NLP problems can be unified under a general sequence model structure, represented by the function $$\mathbf{o} = g(x_0, x_1, ..., x_m; \theta)$$. In this formula, $$x_0, x_1, ..., x_m$$ is the input token sequence, where $$x_0$$ is a special start-of-sequence symbol (such as $$\langle s \rangle$$ or $$\mathbf{[\mathrm{CLS}]}$$). The function $$g(\cdot; \theta)$$ (also written as $$g_{\theta}(\cdot)$$) is a neural network defined by parameters $$\theta$$, and $$\mathbf{o}$$ is the model's output. A common shorthand for the output is $$\mathbf{o} = g_{\theta}(x_0, x_1, ..., x_m)$$.

General Formulation of a Sequence Model

An engineer develops a model to predict house prices using the equation: `price = w1 * sq_ft + w2 * num_bedrooms + b`. The values `w1`, `w2`, and `b` are learned from a dataset to define the model's behavior. For any new house, its `sq_ft` and `num_bedrooms` are fed into the model to get a price prediction. According to the general convention for a parameterized function, `o = g(x_0, x_1, ...; θ)`, which components of the house price equation are the input variables (`x_i`) and which are the parameters (`θ`)? Explain your reasoning.

Identifying Inputs and Parameters in a Model

A junior data scientist is building a model to predict a customer's likelihood of making a purchase. The model uses the customer's `age` and `income` as inputs. The model's behavior is defined by a set of internal values, represented by `θ`. The data scientist documents the model's function as: `likelihood = predict(age, income, θ)`. Based on the standard convention for writing parameterized functions, analyze this notation. Identify any potential ambiguity or deviation from the standard, and explain how you would rewrite it for better clarity.

Critiquing Model Notation

A simple linear model is defined by the equation `y = mx + b`, where the goal is to predict `y` based on a given `x`. If this model is expressed using the general form for a parameterized function, `o = g(x_0; θ)`, which of the following correctly identifies the components?

Learn Before

Related