Input: strs = ["flower","flow","flight"]

Output: "fl"

Solution Logic:

• The first element in the array is taken as a reference, and a loop is established sequentially through the characters of this element.
• At each loop step, the current character is compared with the remaining elements of the array.
• If there is a character mismatch at the same indices or if the character index exceeds the size of the compared word, the process is terminated immediately and the current accumulated prefix up to that point is returned.

TypeScript​

function longestCommonPrefix(strs: string[]): string { 
	let commonPrefix = ''; 
	let i = 0; 
	let hasMatch = true; 
	while (hasMatch) { 
		let current = strs[0][i]; 
		if(!current) return commonPrefix; 
		for(let j=1; j<strs.length; j++) { 
			if(current !== strs[j][i]) return commonPrefix; 
		} 
		commonPrefix += current; i++; 
	} 
	return commonPrefix; 
}

In TypeScript, the return type of functions can be specified by using a : after the parentheses. Specifying types in function parameters is a good practice and usually mandatory. When defining variables, let or const is used; an explicit type (Type Annotation) can be added with : if desired. When no type is specified, TypeScript automatically guesses the variable's type thanks to Type Inference.

JavaScript​

/** 
* @param {string[]} strs 
* @return {string} 
*/ 
var longestCommonPrefix = function(strs) { 
	let commonPrefix = ''; 
	let i = 0; 
	let hasMatch = true; 
	while (hasMatch) { 
		let current = strs[0][i]; 
		if(!current) return commonPrefix; 
		for(let j=1; j<strs.length; j++) { 
			if(current !== strs[j][i]) return commonPrefix; 
		} 
		commonPrefix += current; 
		i++; 
	} 
	return commonPrefix; 
};

This code is a nice longest common prefix solution written in JavaScript. JavaScript and TypeScript syntax are fundamentally very similar; functions can even be assigned to variables in both languages. As a difference, in JavaScript, instead of pure TypeScript types, we can proceed by using JSDoc (@param, @return) as shown above or without specifying any types at all (dynamically).

Python​

class Solution: 
	def longestCommonPrefix(self, strs: list[str]) -> str: 
		commonPrefix = "" 
		for i in range(len(strs[0])): 
			current = strs[0][i] 
			if current is None: 
				return commonPrefix 
			for j in range(1, len(strs)): 
				if i >= len(strs[j]) or current != strs[j][i]: 
					return commonPrefix 
			commonPrefix += current 
		return commonPrefix

In Python, you need to pay close attention to indentation and block structures (spaces are used instead of curly braces {}). for loops are generally written in the form for i in range(...). Although types are not mandatory in Python, a return type can be specified with -> in the function signature, and data types can be specified with : in parameters (Type Hinting). String and list sizes are measured using the len() function.

C#​

public class Solution { 
	public string LongestCommonPrefix(string[] strs) { 
		string commonPrefix = ""; 
		if (string.IsNullOrEmpty(strs[0])) return commonPrefix; 
		if (strs.Length == 1) return strs[0]; 
		for(int i = 0; i < strs[0].Length; i++) { 
			char current = strs[0][i]; 
			if(char.IsWhiteSpace(current)) break; 
			for(int j = 1; j < strs.Length; j++) { 
				if(i >= strs[j].Length || current != strs[j][i]) 
					return commonPrefix; 
			} 
			commonPrefix += current; 
		} 
		return commonPrefix; 
	} 
}

The output type signature is written at the beginning of the function.

PHP​

class Solution { 
	/** 
	* @param String[] $strs 
	* @return String 
	*/ 
	function longestCommonPrefix($strs) { 
		$commonPrefix = ""; 
		$firstItem = $strs[0]; 
		$firstItemLength = strlen($strs[0]); 
		for($i=0; $i<$firstItemLength; $i++) {
			$current = $firstItem[$i]; 
			for($j=1; $j<count($strs); $j++) { 
				if($i >= strlen($strs[$j]) || $current != $strs[$j][$i]) { 
					return $commonPrefix; 
				} 
			} 
			$commonPrefix .= $current; 
		} 
		return $commonPrefix; 
	} 
}

Every variable starts with the $ sign. We do not specify types except within comments. count() is used for array size, and strlen() for string size. Concatenation of two strings is achieved using . instead of +.

Go (Golang)​

func longestCommonPrefix(strs []string) string { 
	commonPrefix := ""; 
	firstItem := strs[0]; 
	firstItemLength := len(strs[0]); 
	for i := 0; i < firstItemLength; i++ {
		current := firstItem[i]; 
		for j := 1; j < len(strs); j++ { 
			if i >= len(strs[j]) || current != strs[j][i] { 
				return commonPrefix; 
			} 
		} 
		commonPrefix += string(current); 
	} 
	return commonPrefix; 
}

Function Definition: Defined with the func keyword, and the return type of the function is specified at the very end of the parentheses.

Variable Declaration: Defined using a short variable declaration with the := operator, similar to mathematical definitions.

Data Types: Array/Slice definitions are made with square brackets like []string.

Syntax Rules (Critical): Using parentheses () in for and if conditions is not mandatory. However, the opening curly brace { must definitely start on the same line; otherwise, the Go compiler throws an error due to the Automatic Semicolon Insertion (ASI) rule.

Size and Characters: Sizes are measured with len(). When accessing a string index, it returns a value of type byte (uint8); therefore, casting it like string(current) is necessary to use it in concatenation operations.

The purpose of today's post is to gather the essential linear algebra prerequisites that support deep learning. It is meant as a summary. Our topic is the fundamental building blocks of linear algebra: scalars, vectors, and matrices.

Scalar & Vector​

A scalar $k \in \mathbb{R}$ is a quantity that can be represented by a real number but has no direction. A vector, on the other hand, is a geometric object that has both a numerical magnitude and a direction. The conventional notation $\mathbf{v} = \begin{bmatrix} x_1 \\ ... \\ x_n \end{bmatrix}$, $x_1, ..., x_n \in \mathbb{R}$ is in the form of a column matrix. The number of rows of a vector indicates the dimension of the space it belongs to, such that $\mathbf{v} \in \mathbb{R}^n$ is an element of the n-dimensional real space.

Vector Addition and Scalar Multiplication​

Matching components are added. Note that vectors of different dimensions cannot be added.

Multiplying a vector by a scalar means multiplying each component by that scalar.

Vector Length (Norm)​

The length (Euclidean norm) of a vector is the square root of the sum of the squares of its components:

$\|\mathbf{v}\| = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2}$

Unit Vector​

Vectors with a length of 1 unit, that is, satisfying the condition $\|\mathbf{v}\| = 1$, are called unit vectors. They are used to represent coordinate axes.

$\mathbf{v} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = x_1\begin{bmatrix} 1 \\ 0 \end{bmatrix} + x_2\begin{bmatrix} 0 \\ 1 \end{bmatrix}$

The equation above is an example of a linear combination of a vector with unit vectors in two-dimensional space.

Geometric Interpretation of Vectors​

$x_1, ..., x_n$ are called coordinate axes. Each coordinate axis measures the length of the vector along that axis. The origin $O=(0, ... , 0) \in \mathbb{R}^n$ is the starting point for all vectors. Therefore, every point in space represents a vector whose starting point is the origin.

Figure 1: An example of vector and point correspondence in two-dimensional space.

Matrix​

Unfortunately, no one can be told what the Matrix is. You have to see it for yourself.

— Morpheus

A matrix is the numerical representation of a linear transformation that keeps the origin fixed, preserves lines as lines, and does not break the parallelism of parallel lines. Formally, for a transformation $T$ to be linear, it must satisfy the following two conditions:

$T(\mathbf{u}+\mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v}), \newline T(k\mathbf{v}) = kT(\mathbf{v})$

This definition covers operations such as rotation, reflection, scaling, shearing, and projection.

Let $A \in \mathbb{R}^{m \times n}$. Then $A$ is a matrix of size $m \times n$, where $m$ is the number of rows and $n$ is the number of columns.

Matrix Addition and Multiplication​

$A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \quad \text{and} \quad B = \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix}$ be two matrices. The addition and multiplication operations are as follows.

The intuition behind defining matrix multiplication this way is as follows: matrix multiplication is the composition of transformations. The expression $AB$ means applying transformation $B$ first and then transformation $A$. This intuition is the key to understanding why successive layers in deep learning are expressed through matrix multiplication. Matrix multiplication is not commutative: in general, $AB \neq BA$. Changing the order of transformations yields a different result.

Transpose of a Matrix​

$\text{For an } m \times n \text{ matrix } A \text{, its transpose } A^{\top} \text{ is an } n \times m \text{ matrix:}$

$A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \end{bmatrix} \quad \Longrightarrow \quad A^{\top} = \begin{bmatrix} a_{11} & a_{21} \\ a_{12} & a_{22} \\ a_{13} & a_{23} \end{bmatrix}$

The general rule is $(A^{\top})_{ij} = A_{ji}$.

Symmetric Matrix​

A square matrix whose transpose equals itself is called a symmetric matrix.

$A = A^{\top} \quad \Longleftrightarrow \quad a_{ij} = a_{ji} \quad \forall\, i, j$

In deep learning, symmetric matrices such as the covariance matrix, Hessian, and Gram matrix are frequently encountered.

Geometric Interpretation of Matrix & Vector​

Multiplying a matrix by a vector applies the linear transformation represented by that matrix to the vector.

Let's work through an example: $\newline A = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix} \in \mathbb{R}^{2 \times 2}$ be a matrix and $\mathbf{v} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} \in \mathbb{R}^{2}$ be a vector.

The new vector resulting from the matrix multiplication $\mathbf{v'} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$ is the linear transformation of $\mathbf{v}$. That is, the vector has been transformed — its direction and magnitude have changed.

Figure 2: The orange vector represents $\mathbf{v}$, and the blue vector represents $\mathbf{v'}$.

Conclusion​

Linear algebra is one of the best tools we have for modeling the spaces we scientifically explore. Fitting all those spaces onto paper is no small feat. If the reader wishes to dive deeper, they can explore the resources below.

See you in the next post!

References​

1. Dive into Deep Learning
2. Essence of Linear Algebra — 3Blue1Brown

See you in other blog posts!