[Openpvrsgx-devgroup] trying to get SGX 1.14 running on DM3730 (SGX530)

H. Nikolaus Schaller hns at goldelico.com
Thu Nov 21 14:28:24 CET 2019


Hi Adam,

> Am 21.11.2019 um 14:15 schrieb Adam Ford <aford173 at gmail.com>:
> 
> On Thu, Nov 21, 2019 at 5:58 AM H. Nikolaus Schaller <hns at goldelico.com> wrote:
>> 
>> Hi,
>> 
>>> Am 20.11.2019 um 20:14 schrieb H. Nikolaus Schaller <hns at goldelico.com>:
>>> 
>>> Hi,
>>> I spent a little time to better analyse what is going
>>> wrong when using the am335x UM code on the DM3730.
>>> 
>>> Not much...
>>> 
>>> pvrsrvctl --start --no-module
>>> 
>>> works fine as expected.
>>> 
>>> Then, running gles1test1 fails with no primary display.
>>> 
>>> I did an strace and found that it is calling the
>>> DRM_IOCTL_VERSION ioctl for all 16 cards between
>>> /dev/dri/card0 and /dev/dri/card15.
>>> 
>>> It finds as expected card0 (the SGX) and card1 (the DISPC).
>>> 
>>> First I though that the cards may be swapped - Tony mentioned
>>> that as a risk. But they are exactly the same as on BeagleBone
>>> black.
>>> 
>>> On the BBB it does the same ioctl on /dev/dri/card0 and succeeds.
>>> 
>>> There are no other syscalls involved. Just open(), ioctl() and
>>> close().
>>> 
>>> On one architecture it is found on the other it isn't.
>>> 
>>> So I tried to find a difference. The ioctl is handled in
>>> drivers/gpu/drm/drm_ioctl.c so I added a prink to the handler
>>> to show what the DRM subsystem is reporting to user-space.
>>> 
>>> It is exactly the same:
>>> 
>>> Pyra: (OMAP5)
>>> 
>>> [   53.013185] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>>> 
>>> vs. GTA04 (DM3730)
>>> 
>>> [  144.491760] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>>> 
>>> vs. BBB (AM335x)
>>> 
>>> [   68.505033] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>>> 
>>> So there must be some other factor for which the loop over
>>> the dri/cards does not find something.
>>> 
>>> What I could imagine (but is difficult to test) is that
>>> the strings printed by printk do not properly arrive in
>>> user-space.
>>> 
>>> I.e. if drm_copy_field leaves some user-space bytes uninitialized
>>> and the user-space code still checks for them. It might have a bug
>>> and e.g. expect a \0 for the strings although the length is passed.
>>> This could be a strcmp() instead of a strncmp().
>>> 
>>> Or there is some other factor tested before the gles1test1
>>> does run the ioctl() loop but only manifests itself after
>>> doing the ioctl().
>>> 
>>> BTW: pvrsrvctl also calls this ioctl twice and seems to
>>> successfully find the card... This would also be an indicator
>>> for user-space memory corruption or missing initialization.
>>> 
>>> So at the moment I have no idea how to find out more.
>> 
>> Some more findings.
>> 
>> 1. here is the dmesg log from DM3730:
>> 
>> root at letux:~# pvrsrvctl --start --no-module
>> [  250.508331] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  250.519531] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  250.534362] PVR_K: UM DDK-(3699939) and KM DDK-(3699939) match. [ OK ]
>> root at letux:~# strace gles1test1 >gta04.txt 2>&1
>> [  275.720184] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  275.731597] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  275.750885] DoMapToUser: pfn=0004804a vmf_insert_mixed(vma=dae1b6c0, addr=b6f26000, pfn_t=4804a)  ret=00000100
>> [  275.763031] PVR_K:(Error): DoMapToUser: Error - vmf_insert_mixed failed (100)
>> [  275.885498] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  275.896789] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [  275.910827] drm_version: maj=1 min=0 patch=0 name=omapdrm date=20110917 desc=OMAP DRM
>> [  275.920928] drm_version: maj=1 min=0 patch=0 name=omapdrm date=20110917 desc=OMAP DRM
>> root at letux:~#
>> 
>> and from am335x:
>> 
>> root at letux:~# pvrsrvctl --start --no-module
>> [   45.811912] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [   45.823160] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [   45.839198] PVR_K: UM DDK-(3699939) and KM DDK-(3699939) match. [ OK ]
>> root at letux:~# strace gles1test1 >bbb.txt 2>&1
>> [   84.837114] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [   84.848381] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [   84.865905] DoMapToUser: pfn=0004804a vmf_insert_mixed(vma=dd71f900, addr=b6f42000, pfn_t=4804a)  ret=00000100
>> [   84.878029] PVR_K:(Error): DoMapToUser: Error - vmf_insert_mixed failed (100)
>> [   84.982602] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
>> [   84.992450] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
>> [   85.004594] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
>> [   85.015917] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
>> [   85.027867] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> [   85.040403] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
>> root at letux:~#
>> 
>> No significant difference.
>> 
>> The assignment of card0 / card1 to display driver and SGX is different, but doesn't matter
>> (I have tested by blacklisting the SGX driver and modprobing manually).
>> 
>> 2. Analysis of the strace shows that the sequence of events is
>> 
>> a) drm_version called twice on both platforms
>> 
>> open("/dev/dri/renderD128", O_RDWR|O_LARGEFILE) = 3
>> ioctl(3, DRM_IOCTL_VERSION, 0x43a738)   = 0
>> ioctl(3, DRM_IOCTL_VERSION, 0x43a738)   = 0
>> ioctl(3, DRM_IOCTL_I810_FLUSH or DRM_IOCTL_MGA_SWAP or DRM_IOCTL_R128_CCE_RESET or DRM_IOCTL_RADEON_CP_RESET, 0) = -1 EACCES (Permission denied)
>> fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf69c) = 0
>> getpid()                                = 2526
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf56c) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf6ac) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf674) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5a4) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf6e4) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf3dc) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5ec) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5c4) = 0
>> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf474) = 0
>> 
>> b) then, the DoMapToUser error happens but does not abort
>> 
>> c) then all /dev/dri/card* are scanned
>> 
>> open("/dev/dri/card0", O_RDWR)          = 4
>> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f20)   = 0
>> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f20)   = 0
>> close(4)                                = 0
>> open("/dev/dri/card1", O_RDWR)          = 4
>> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f70)   = 0
>> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f70)   = 0
>> close(4)                                = 0
>> open("/dev/dri/card2", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card3", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card4", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card5", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card6", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card7", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card8", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card9", O_RDWR)          = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card10", O_RDWR)         = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card11", O_RDWR)         = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card12", O_RDWR)         = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card13", O_RDWR)         = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card14", O_RDWR)         = -1 ENOENT (No such file or directory)
>> open("/dev/dri/card15", O_RDWR)         = -1 ENOENT (No such file or directory)
>> 
>> vs. successful
>> 
>> open("/dev/dri/card0", O_RDWR)          = 4
>> ioctl(4, DRM_IOCTL_VERSION, 0x43df20)   = 0
>> ioctl(4, DRM_IOCTL_VERSION, 0x43df20)   = 0
> 
> Right here is where it appears to change.  Any ideas what the
> difference is between card0 on the different platforms?
> 
>> ioctl(4, DRM_IOCTL_SET_VERSION, 0xbebdfabc) = 0
> 
> Above it closed card0 and started to open card1 indicating to me it
> didnt' find what it wanted.  Do you know what it wanted?

Not yet... This is what I try to find out next by adding printf to libdrm.

> 
>> ioctl(4, DRM_IOCTL_VERSION, 0x43df68)   = 0
>> ioctl(4, DRM_IOCTL_VERSION, 0x43df68)   = 0
>> open("/dev/dri/renderD128", O_RDWR|O_LARGEFILE) = 5
> 
> Looking at your logs (c), the above doesn't show it trying to open
> renderD128, but the log in (a) seems to start with opening renderD128.
> I am not very familiar with the 3D stuff, but
>> ioctl(5, DRM_IOCTL_VERSION, 0x43df20)   = 0
>> ioctl(5, DRM_IOCTL_VERSION, 0x43df20)   = 0
>> ioctl(5, DRM_IOCTL_I810_FLUSH or DRM_IOCTL_MGA_SWAP or DRM_IOCTL_R128_CCE_RESET or DRM_IOCTL_RADEON_CP_RESET, 0) = -1 EACCES (Permission denied)
>> fcntl64(5, F_SETFD, FD_CLOEXEC)         = 0
>> 
>> 
>> 3. I have searched for the source code and things seem to fail in libdrm - which
>> is good, because it is open source :)
> 
> I was investigating some issues with the Etnaviv driver on the i.MX6 a
> while back, and there was an issue where the DMA driver changed the
> available memory.  For while, I created a reserved memory pool in the
> device tree which fixed the issue until a permanent solution was
> found.  I haven't looked at the full device tree of the am33xx you're
> using, but do you know if it's using any sort of reserved memory pool?
> If there is a difference between the am33xx and dm3730, it might be
> something to investigate.

What I have tried is to play with coherent_pool= on the kernel command line
but 1M or 8M doesn't make a difference. It seems to be needed 1M to make
the am335x work. It is not needed on the omap5 (at least my setup).

So such defaults and requirements may make a difference.

> 
>> 
>> First of all calling DRM_IOCTL_VERSION is to get the string lengths by the first
>> call to be able to allocate string buffers for the second call.
>> 
>> The code looks sane and takes care about length and extra \0 for C strings.
>> 
>> So next step will be to build my own version from sources and add some test code
>> to understand why the search for the SGX fails. It would also be possible to use
>> gdb gles1test1 and single-step through drmGetVersion() but that also needs source
>> code...
>> 
>> Anyways there are interesting parts in libdrm-2.4.74/xf86drm.c
> 
> Are you only trying the 3D in Xf86?

Well, I am not even using X. It just seems to be the source file where the drmGetVersion()
comes from that fails.

So let's wait what I find out by digging into libdrm2. Just got the first
libdrm2-dbg_2.4.74-2hns_armhf.deb
compiled. Now have to find out how to install and if it still works...

BR,
Nikolaus



More information about the openpvrsgx-devgroup mailing list