LBP

LBP

1. Texture

Def:

• Includes: more regular patterns

• Includes: more random patterns

2. Texture-related tasks

• Shape from texture
  • Estimate surface orientation方向 or shape from image texture
• Segmentation/classification from texture cues
  • Analyze, represent texture
  • Group image regions with consistent texture
• Synthesis合成
  • Generate new texture patches/images given some examples

3.Why analyze texture?

Importance to perception:

• Often indicative of a material’s properties 表示一种材料的特性

• Can be important appearance cue, especially if shape is similar across objects

• Aim to distinguish between shape, boundaries, and texture

Technically:

• Representation-wise, we want a feature one step above “building blocks” of filters, edges.

4. Texture representation

• Textures are made up of repeated local patterns, so:
  • Find the patterns
    • Use filters that look like patterns (spots, bars, …)
    • Consider magnitude of response
  • Describe their statistics within each local window, e.g., 用数值特征表示
    • Mean, standard deviation
    • Histogram
    • Histogram of “prototypical” feature occurrences

4.1 Texture representation: example

• 图中，分别使用x梯度算子与y梯度算子，对每个图像中的pattern进行卷积，然后取这个pattern的平均值作为输出

• 将其投射到坐标系，可以得到，靠近原点即为平滑区域，远离则为角点

• 坐标系中也可以看出纹理的区别

• Distance reveals how dissimilar texture from window a is from texture in window b.

4.2 window scale

• We’re assuming we know the relevant window size for which we collect these statistics.

• 用不同窗口的size，提取不同尺度的纹理，类似SIFT，遍历所有size

• Possible to perform scale selectionby looking for window scale where texture description not changing.

5. Filter banks

• Our previous example used two filters, and resulted in a 2-dimensional feature vector to describe texture in a window.
  • x and y derivatives revealed something about local structure.
• We can generalize to apply a collection of multiple (d) filters: a “filter bank”
• Then our feature vectors will be d-dimensional.
  • still can think of nearness, farness in feature space

5.1 What filters to put in the bank?

• Typically we want a combination of scales and orientations, different types of patterns. 由检测不同尺度与方向的特征组合

5.1.1 Multivariate Gaussian

Filter bank：

Can you match the texture to the response?

5.2 Representing texture by mean abs response

• We can form a feature vector from the list of responses at each pixel.

5.3 d-dimensional features

• Euclidean distance ($L_2$​)

• 跟这六类比较，使用SVM分类

6. Local Binary Pattern (LBP)

• 用中心像素与领域像素的大小关系来表示其特征，最终化为二值向量
  • Use center pixel value to threshold the 3x3 neighborhood 用中心像素阈值，因为比它小的直接就变成零了
  • Result in binary number 转化为二进制向量
  • Multiplied by powers of two (Decimal) 转化为十进制，以缩短长度
  • Summed to obtain a label for the center pixel -> 256 different labels（每个window都用1*8的二进制数表示，其范围为0-255）
  • Histogram of the labels is used as a texture descriptor 用LBP直方图统计label个数，然后用1*256的向量表示一张图片
  • 从而使得一张20002000的图片变成1\256

6. 1 What are the problems? How can you be invariant to changes in scale ?

• 没有考虑不同的尺度空间。

解决方法

6.2 Circle LBP

• LBP is extended to use different sizes of neighborhoods.
• Local neighborhoods is defined as a set of sampling points.
• points evenly （平均） spaced on a circle centered at the labeled pixel.
• (P,R) , P = number of sampling points , R = radius

6.3 Uniform LBP

• Standard LBP has $2^n$​​ patterns for n sampling points 因为最终会表示成二进制数的直方图
  • Histogram dimension becomes high when n is increased
  • Sensitive to noise

• Uniform patterns has at most 2 bitwise transitions in binary pattern. 当0变成1就被噪音影响了

• Histogram assigns separate bin for every uniform pattern.

• Histogram assigns a single bin for all non-uniform pattern.

• In FERET dataset, (8,1) neighborhoods : 90.6 percent of patterns are uniform.

• 特征空间还是过长了

• LBP是比较中间像素与领域像素的比较，但如果有噪声1就变成0了，即对噪声过于敏感了
• 如果有大于两次跳变，就是不平滑，所以我们把所有超过两次跳变的归为一类，在直方图统一计算bin，而小于等于两次跳变的，即为uniform LBP，这些可以分开计数，从而压缩噪声

步骤：

• Step 1: facial image is divided into local regions (blocks). {R0, R1, …, Rm-1} (pixel-level locality)

• Step 2: Extract LBP histogram for each region. (regional-level locality)

• Step 3: Concatenated（连接） all histograms into a spatially enhanced histogram with length of m x n (n is length of a single LBP histogram). (global-level locality)

  • m blocks

• Step 4: Chi-square distance

7. Summary

• Texture is a useful property that is often indicative of materials, appearance cues
• Texture representations attempt to summarize repeating patterns of local structure
• Filter banks useful to measure redundant（冗余的） variety of structures in local neighborhood
  • Feature spaces can be multi-dimensional
• Local binary patterns describe small-scale appearance (textures) of the image