MiPi CSI(Rockchip Linux)

|

RK平台上android使用的MIPI CSI驱动是userspace based的驱动, 需要基于android的hal层使用,因此并不适合linux平台.

按照linux standard, 需要是写成v4l2形式的驱动, 类似uvc, 从而可以兼容常见的应用, 比如opencv, gstreamer……

RK的v4l2驱动也分两套.

一套叫cif_isp10, 用于rv系列和sofia系列的linux系统. 我个人来说, 并不是很喜欢这套驱动, 因为他只是output的接口用了v4l2而已, 但内部的实现比较蛋疼. 一是sensor驱动的框架是自己实现了一套,并没有基于标准的subdev接口, 调试sensor的时候不能使用已有的在driver/media/i2c下的驱动. 二是模块分层并不清晰, 理解代码和定位流程都很麻烦.

另一套叫rockchip-isp1, 是后来在chromeos项目上基于cif_isp10改出来的, 符合linux standard的驱动, 也是我们现在给3288使用的驱动.
这个驱动有了比较多的review, 结构比较清晰.

拓扑接口:
sensor->mipiphy->isp subdev->sp/mp vdev

如下的硬件框图:
3的输出对应mp vdev(/dev/videox+2), 可以输出raw格式和yuv格式的数据.
4的输出对应sp vdev(/dev/videox), 可以输出yuv格式和rgb格式的数据.
1-crop, 2-resize, 3和4全部都在capture.c里, 对外stream vdev的形式, 就是你看到的/dev/videox, 对这个节点s_fmt调用resize, s_selesction调用crop.
isp, ie, si是用于3a, 图像处理的部分, 被隐藏在isp subdev里, 默认下输出的是未处理的图像, 只有使用isp gstreamer plugin或者我们提供的linux hal时, 会操作这个subdev, 传入tuning的数据.

Tips

sensor

调试sensor的技巧:

第一步是在rkisp1_mipi_isr里加中断, 确认mipi phy的信号有出来. mipi phy有信号, 可以验证以下事情: mipi phy/controler可以接收输入, sensor信号有了输出. 反之如果没有, 原因也就是这个两个, 可以用下面的命令来验证.

io -4 -l 0x100 0xff911c00(rk3288)

出来的数据, 第2个部分代表mipi status, 多输入几次, 看看有没有变化, bit 8:11代表data lane0-3的stop state, bit12代表clock lane stop state. 如果有所的stop bit没变, 需要检查sensor是不是正确输出了, 如果只是clock lane, 需要看下sensor的clock lane是不是在s_stream前没在lp11模式.

第二步, 如果mipi_isr有中断了, 但是有sync_error的错误, 表示mipi接收的图像和isp配置的信息不一致, 可以有以下原因: mipi接收的图形大小和isp配置的不一致, mipi接收的信号有问题.
第一个原因,检查下是不是sensor配置了1080p输出, 但是设置的软件size确是720p.
第二个原因,检查下是不是bitrate设置的有问题, lane num设置的有问题, 硬件信号有问题.

Burn Mac Address(Rockchip Linux)

|

Unlike rockchip android, we don’t limit you how to get mac address. You can write/read it in any places.

Usually, it could be stored in reserved1.
http://opensource.rock-chips.com/wiki_Partitions

Bellow is how to burn mac address to reserved1 and read it.

First, enter maskrom and issue fllowing command to burn mac adress.

printf '\xde\xad\xbe\xef\xef\xee' > mac.bin
build/flash_tool.sh -p reserved1 -c rk3288 -i mac.bin

Then, add ethadder-set fucntion to u-boot board file, it will read mac address from reserved1.

#include <mmc.h>

int rk_board_late_init(void)
{
  u8 ethaddr[1024];

  // 1 for sd-card
  struct mmc *mmc = find_mmc_device(0);
  struct blk_desc *desc = mmc_get_blk_desc(mmc);
  unsigned long blk_start = 8064;
  unsigned long blk_cnt = 1;

  int d = blk_dread(desc, blk_start, blk_cnt, ethaddr);

  eth_setenv_enetaddr("ethaddr", ethaddr);

  return 0;
}

Show Boot Logo(Rockchip Linux)

|

Unlik rockchip android, we don’t have a complete method for boot logo on linux.

It means you can only show logo by generic way, and there are three parts.

U-boot

First, you need enable video driver in u-boot and make it show console in screen. and then, apply below patches.(Don’t forget to remove console in screen)

https://github.com/wzyy2/Scripts/blob/master/tmp/U-Boot-v1-1-2-board-rockchip-common-Add-bmp-logo-support.patch
https://github.com/wzyy2/Scripts/blob/master/tmp/U-Boot-v1-2-2-rockchip-include-Enable-logo-display-for-evb-rk3399.patch

Since rockchip rom code have litmit the uboot size to 1Mb, so you can’t display with big size bmp. If you want to display big size bmp, you might need use gzipped bmp.

https://github.com/rockchip-linux/u-boot/blob/release/cmd/bmp.c#L40

Kernel

http://blog.csdn.net/wang_zheng_kai/article/details/41847687

set logos_freed to false
http://elixir.free-electrons.com/linux/latest/source/drivers/video/logo/logo.c#L32

System

You can use ‘plymouth’ to show logo before desktop launched.

If you are using debian, there are some bug need fix.

  1. Change ExecStartPost in /lib/systemd/system/plymouth-start.service to ExecStartPost=-/usr/bin/udevadm settle –timeout=30 –exit-if-exists=/sys/class/drm/card0/dev ; /usr/bin/udevadm settle –timeout=30 –exit-if-exists=/sys/class/graphics/fb0/dev ;

  2. Add plymouth-quit.service to After in /lib/systemd/system/lightdm.service

Swap video layer(Rockchip Linux)

|

By default, Video layer is on top of GUI layer, which means you can’t put GUI items on top of video.

To allow video work with GUI, there are two ways.

  1. Use Wayland dma-buf api
    If there is something cover the video layer, wayland compositor will use GPU to composite. The advantage is that it’s a standard and the api is clean. The disadvantage is that wayland have many compatibility problem(e.g QT) and the using of GPU will reduce performance.

  2. Change video layer to the bottom
    In this way, we could make the place we want to show video transparent, so the bottom layer would show up. The advantage is that it has the best performance. The disadvantage is that it’s kind of hacky.

In below, i will introduce how to enable the second way on different display server.

First, you need apply the below kernel patches.

The Kernel Patch for swap layer:
kernel patch 1

The Kernel Patch for dsiable double vsync with drm api:
kernel patch 2

QT eglfs

“we could make the place we want to show video transparent”
To make qt background transparent, we need this patches to change the default color format to ARGB.
qt patch 1

Then set the alpha value of the window you want to show video to zero, VOP will skip them when do alpha blending.

display

Before starting application, please set below env value to force qt use kmssink to show video on bottom.

export QT_GSTREAMER_WINDOW_VIDEOSINK=kmssink

qt class

Please use qmlvideo

X11/Wayland

On X11 and Wayland backend, we can’t make GUI layer transparent by alpha becuase of the exists of desktop background. If you set window backgournd color to transparent, it would show background rather than video lyaer, so we can only use color key.

To set color key, it need some changes in kernel driver.

kernel patch 5

kernel patch 6

it will make black color a transparency color key.

display

Before starting application, please set below env value to force qt use rkximagesink to show video on bottom.

export QT_GSTREAMER_WINDOW_VIDEOSINK=rkximagesink

qt class

qmlvideo, qtvideowidget

Others

Somtimes we need a 1080P UI with the video layer in 4K resolution for smooth, then we might need this patch.

The Kernel Patch for 1080P UI + 4K video layer :
kernel patch 3

The Qt patch for 1080 UI:
kernel patch 4

Rockchip Linux Tips

|

这里主要记录一些个人的测试命令和额外的补丁。

spi

https://github.com/rockchip-linux/kernel/blob/release-4.4/drivers/spi/spidev.c

如果你想要有spidev出现在sysfs下,你需要在dts里注册这个spidev!

bus {
  spidev ...
}

bus是bus,dev是dev,spidev作为虚拟的设备,也是必须要声明这个dev才会被probe。

i2c

i2c就有一个方便的sys接口了,可以看下面的文章 https://erlerobotics.gitbooks.io/erle-robotics-erle-brain-a-linux-brain-for-drones/en/tutorials/i2c.html

wifi

见 RK WIFI 文档 

usb

见 RK USB Compliance Test Note V1.2.pdf

eth

可以用下面办法直接写寄存器

    cd /sys/bus/mdio_bus/devices/stmmac-0\:01
    echo 20 > phy_reg
    echo 0x400 > phy_regValue

单通道ddr 3288

休眠补丁 https://github.com/wzyy2/Scripts/blob/master/tmp/3288_fix_ddr_single_channel.patch

uboot制作命令 dd if=tools/rk_tools/bin/rk32/32_LPDDR2_400MHz_LPDDR3_400MHZ_DDR3_400MHZ_20160803.bin of=spl.bin bs=4 skip=1 && tools/mkimage -n rk3288 -T rksd -d spl.bin rk3288_uboot.img && cat u-boot-dtb.bin » rk3288_uboot.img

Comments