Detailed YUV format, detailed explanation of YUV420 data format

The YUV format has two broad categories: planar and packed. For the planar YUV format, the Y of all pixels is stored consecutively, followed by the U of all the pixels, followed by the V of all the pixels. For the packed YUV format, Y, U, and V of each pixel are consecutively *stored.

YUV is divided into three components, "Y” represents brightness (Luminance or Luma), which is the gray value; and "U" and "V"" is the color ( Chrominance or Chroma), which describes the color and saturation of the image, and is used to specify the color of the pixel.

Like similarly known RGB, YUV is also a color coding method, mainly used in television systems and analog video, which separates luminance information (Y) from color information (UV) without UV information. The complete image can be displayed, just black and white, which is a good solution to the compatibility of color TVs with black and white TVs. Moreover, YUV does not require three independent video signals to be transmitted at the same time as RGB, so the YUV mode transmits very little bandwidth.

The storage format of YUV stream is closely related to the way of sampling. There are three mainstream sampling methods, YUV4:4:4, YUV4:2:2, and YUV4:2:0. I can find out through other articles on the Internet. Here I want to emphasize how to restore the YUV value of each pixel from the code stream according to its sampling format, because only the YUV value of each pixel can be restored correctly, and the YUV can be passed through YUV. The RGB conversion formula extracts the RGB values ​​of each pixel and displays them.

Use three graphs to visually indicate the way of acquisition. The black component indicates the Y component of the pixel, and the open circle indicates the UV component of the pixel.
Yuv format detailed

First remember the following paragraph, and then extract the YUV component of each pixel will be used later.

1. YUV 4:4:4 sampling, each Y corresponds to a set of UV components.
   
2. YUV 4:2:2 sampling, each pair of Y shares a set of UV components.
   
3. YUV 4:2:0 sampling, every four Y shares a set of UV components.
   
   

   2. Storage Method
   

   The following is a way to store the common YUV code stream in the form of a graph, and a YUV that samples each pixel is attached to the storage method. The method of data, wherein the meanings of Cb and Cr are equivalent to U and V.
   

   (1) YUVY format (belongs to YUV422)
   YUVY format
   

   YUYV is one of the storage formats for YUV422 sampling, and two adjacent Ys share their adjacent two For Cb and Cr, the values ​​of Cb and Cr are Cb00 and Cr00 for pixel points Y'00 and Y'01, and the YUV values ​​of other pixels are analogous.
   

   (2) UYVY format (belongs to YUV422)
   UYVY format
   

   The UYVY format is also one of the storage formats for YUV422 sampling, except that the order of UV is different from YUYV. The difference is that the method of restoring the YUV value of each pixel is the same as above.
   

   (3) YUV422P ( belongs to YUV422)
   YUV422P
   

   YUV422P is also a kind of YUV422, which is a Plane mode, ie flat mode, which does not interleave YUV data. Instead, store all the Y components first, then store all the U(Cb) components, and finally store all the V(Cr) components, as shown in the figure above. The YUV value extraction method for each pixel is also the most basic extraction method according to the YUV422 format, that is, two Ys share one UV. For example, for the pixel points Y'00 and Y'01, the values ​​of Cb and Cr are Cb00 and Cr00.
   

   (4)YV12, YU12 format (belongs to YUV420)
   YV12, YU12 format
   

   YU12 and YV12 belong to YUV420 format, and also a Plane mode, which will Y, U, V components Packed separately and stored in order. The YUV data extraction of each pixel follows the extraction method of the YUV420 format, that is, the four Y components share a set of UVs. Note that in the above figure, Y'00, Y'01, Y'10, and Y'11 share Cr00 and Cb00, and so on.
   

   (5)NV12, NV21 (belongs to YUV420)
   NV12, NV21
   

   NV12 and NV21 belong to the YUV420 format, which is a two-plane mode, that is, Y and UV are divided into two. Plane, but UV (CbCr) is interlaced, not divided into three planes. The extraction method is similar to the previous one, that is, Y'00, Y'01, Y'10, Y'11 share Cr00, Cb00
   

   YUV420 planar data, taking 720× 488 size image YUV420 planar as an example ,
   

   The storage format is: Total size is (720× 480× 3>>1) bytes,
   

   is divided into three parts: Y, U and V
   

   Y component: (720× 480) bytes
   

   U(Cb) component: (720× 480>> 2) bytes
   

   V(Cr) component: ( 720× 480>>2) Bytes
   

   The three parts are internally stored in line priority, and the three parts are sequentially stored in Y, U, and V.
   

   is 0--720× of YUV data; 480 bytes are Y component values,
   

   720× 480--720× 480× 5/4 bytes are U components
   

   720×480×5/4 --720×480× 3/2 bytes are V components.
   

   4 :2: 2 and 4:2:0 Conversion:
   

   The easiest way:
   

   YUV4:2:2 ---> YUV4:2: 0 Y is unchanged, and the U and V signal values ​​are subjected to interlaced sampling in the row (vertical direction). YUV4:2:0 ---> YUV4:2:2 Y does not change, copy each line of U and V signal values ​​to form two consecutive lines of data.
   

   In the YUV420, one pixel corresponds to one Y, and one 4X4 small square corresponds to one U and V. For all YUV420 images, their Y-value alignment is exactly the same, because only the Y image is a grayscale image. The data format of YUV420sp and YUV420p their UV arrangement is completely different in principle. 420p It is to store U after storage, then store V, which means that they are continuous. The 420sp is alternately stored in UV and UV. (See the figure below) With the above theory, I can accurately calculate the size of a YUV420 stored in memory. Width * hight = Y (sum) U = Y /4 V = Y /4
   

   So the length of the YUV420 data in memory is width * hight * 3 /2,
   

   assuming a resolution The rate is 8X4 YUV images, their format is as follows:
   

   YUV420sp format as shown below
   YUV420sp format diagram
   

   YUV420p data format as shown below
   YUV420p data format diagram
   

   Rotate 90 degrees algorithm:
   

   public static void rotateYUV240SP(byte[] src,byte[] des,int width,int height){int wh = width * height;//rotate Yint k = 0;for(int i=0;i<width;i++){for(int j=0;j<height;j++){des[k] = src[width*j + i];k++;}}for( Int i=0;i<width;i+=2){for(int j=0;j<height/2;j++){des[k] = src[wh+ width*j + i];des[k+1 =============== RGB24 format, RGB24 frame size size=width×heigth×3 Bit, RGB32 size=width×heigth×4, if it is I420 (ie YUV standard format 4:2:0) the amount of data is size=width× Heigth× 1.5 Bit. After the RGB24 data is acquired, the data of this format needs to be compressed for the first time. The color space of the image is made up of RGB2YUV. Because X264 requires standard YUV (4:2:0) when encoding. However, it should be noted here that although YV12 is also (4:2:0), YV12 and I420 are different, and there are some differences in the storage space. As follows: YV12: Brightness (row & times; column) + U (row & times; column /4) + V (row & times; column /4)
   

   I420 : Brightness (row & times; column) + V (row × column /4) + U (row & times; column /4)
   

   It can be seen that YV12 and I420 are basically the same, that is, the order of UV is different.
   

   Continue our topic, after the first data compression, RGB24->YUV(I420). In this way, the amount of data will be reduced by half. Why? Oh, this is too basic, I will not write more. Similarly, if it is RGB24->YUV (YV12), it is also reduced by half. However, although it is half, if it is YV12, the effect will be greatly lost. Then, after X264 encoding, the amount of data will be greatly reduced. The encoded data is packaged and transmitted in real time through RTP. After arriving at the destination, the data is taken out and decoded. After decoding, the data is still in YUV format, so you need to convert once, so the windows driver can handle it, which is YUV2RGB24.
   

   YUY2 is 4:2:2 [Y0 U0 Y1 V0]
   The difference between yuv420p and YUV420 differs in storage format yuv420p:yyyyyyyy uuuuuuuu vvvvv yuv420: yuv yuv yuv YUV420P,Y,U,V The three components are all in flat format and are divided into I420 and YV12. The difference between the I420 format and the YV12 format is different in the U plane and the V plane. In the I420 format, the U plane follows the Y plane and then the V plane (ie: YUV); but YV12 is the opposite (ie: YVU). YUV420SP, Y component plane format, UV packing format, ie NV12. NV12 is similar to NV21, with U and V staggered, differing in UV order. I420: YYYYYYYY UU VV => YUV420PYV12: YYYYYYYY VV UU => YUV420PNV12: YYYYYYYY UVUV => YUV420SPNV21: YYYYYYYY VUVU => YUV420SP