[Openpvrsgx-devgroup] trying to get SGX 1.14 running on DM3730 (SGX530)

Adam Ford aford173 at gmail.com
Thu Nov 21 14:15:51 CET 2019


On Thu, Nov 21, 2019 at 5:58 AM H. Nikolaus Schaller <hns at goldelico.com> wrote:
>
> Hi,
>
> > Am 20.11.2019 um 20:14 schrieb H. Nikolaus Schaller <hns at goldelico.com>:
> >
> > Hi,
> > I spent a little time to better analyse what is going
> > wrong when using the am335x UM code on the DM3730.
> >
> > Not much...
> >
> > pvrsrvctl --start --no-module
> >
> > works fine as expected.
> >
> > Then, running gles1test1 fails with no primary display.
> >
> > I did an strace and found that it is calling the
> > DRM_IOCTL_VERSION ioctl for all 16 cards between
> > /dev/dri/card0 and /dev/dri/card15.
> >
> > It finds as expected card0 (the SGX) and card1 (the DISPC).
> >
> > First I though that the cards may be swapped - Tony mentioned
> > that as a risk. But they are exactly the same as on BeagleBone
> > black.
> >
> > On the BBB it does the same ioctl on /dev/dri/card0 and succeeds.
> >
> > There are no other syscalls involved. Just open(), ioctl() and
> > close().
> >
> > On one architecture it is found on the other it isn't.
> >
> > So I tried to find a difference. The ioctl is handled in
> > drivers/gpu/drm/drm_ioctl.c so I added a prink to the handler
> > to show what the DRM subsystem is reporting to user-space.
> >
> > It is exactly the same:
> >
> > Pyra: (OMAP5)
> >
> > [   53.013185] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> >
> > vs. GTA04 (DM3730)
> >
> > [  144.491760] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> >
> > vs. BBB (AM335x)
> >
> > [   68.505033] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> >
> > So there must be some other factor for which the loop over
> > the dri/cards does not find something.
> >
> > What I could imagine (but is difficult to test) is that
> > the strings printed by printk do not properly arrive in
> > user-space.
> >
> > I.e. if drm_copy_field leaves some user-space bytes uninitialized
> > and the user-space code still checks for them. It might have a bug
> > and e.g. expect a \0 for the strings although the length is passed.
> > This could be a strcmp() instead of a strncmp().
> >
> > Or there is some other factor tested before the gles1test1
> > does run the ioctl() loop but only manifests itself after
> > doing the ioctl().
> >
> > BTW: pvrsrvctl also calls this ioctl twice and seems to
> > successfully find the card... This would also be an indicator
> > for user-space memory corruption or missing initialization.
> >
> > So at the moment I have no idea how to find out more.
>
> Some more findings.
>
> 1. here is the dmesg log from DM3730:
>
> root at letux:~# pvrsrvctl --start --no-module
> [  250.508331] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  250.519531] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  250.534362] PVR_K: UM DDK-(3699939) and KM DDK-(3699939) match. [ OK ]
> root at letux:~# strace gles1test1 >gta04.txt 2>&1
> [  275.720184] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  275.731597] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  275.750885] DoMapToUser: pfn=0004804a vmf_insert_mixed(vma=dae1b6c0, addr=b6f26000, pfn_t=4804a)  ret=00000100
> [  275.763031] PVR_K:(Error): DoMapToUser: Error - vmf_insert_mixed failed (100)
> [  275.885498] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  275.896789] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [  275.910827] drm_version: maj=1 min=0 patch=0 name=omapdrm date=20110917 desc=OMAP DRM
> [  275.920928] drm_version: maj=1 min=0 patch=0 name=omapdrm date=20110917 desc=OMAP DRM
> root at letux:~#
>
> and from am335x:
>
> root at letux:~# pvrsrvctl --start --no-module
> [   45.811912] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [   45.823160] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [   45.839198] PVR_K: UM DDK-(3699939) and KM DDK-(3699939) match. [ OK ]
> root at letux:~# strace gles1test1 >bbb.txt 2>&1
> [   84.837114] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [   84.848381] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [   84.865905] DoMapToUser: pfn=0004804a vmf_insert_mixed(vma=dd71f900, addr=b6f42000, pfn_t=4804a)  ret=00000100
> [   84.878029] PVR_K:(Error): DoMapToUser: Error - vmf_insert_mixed failed (100)
> [   84.982602] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
> [   84.992450] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
> [   85.004594] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
> [   85.015917] drm_version: maj=1 min=0 patch=0 name=tilcdc date=20121205 desc=TI LCD Controller DRM
> [   85.027867] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> [   85.040403] drm_version: maj=1 min=14 patch=3699939 name=pvr date=20110701 desc=Imagination Technologies PVR DRM
> root at letux:~#
>
> No significant difference.
>
> The assignment of card0 / card1 to display driver and SGX is different, but doesn't matter
> (I have tested by blacklisting the SGX driver and modprobing manually).
>
> 2. Analysis of the strace shows that the sequence of events is
>
> a) drm_version called twice on both platforms
>
> open("/dev/dri/renderD128", O_RDWR|O_LARGEFILE) = 3
> ioctl(3, DRM_IOCTL_VERSION, 0x43a738)   = 0
> ioctl(3, DRM_IOCTL_VERSION, 0x43a738)   = 0
> ioctl(3, DRM_IOCTL_I810_FLUSH or DRM_IOCTL_MGA_SWAP or DRM_IOCTL_R128_CCE_RESET or DRM_IOCTL_RADEON_CP_RESET, 0) = -1 EACCES (Permission denied)
> fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf69c) = 0
> getpid()                                = 2526
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf56c) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf6ac) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf674) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5a4) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf6e4) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf3dc) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5ec) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf5c4) = 0
> ioctl(3, _IOC(_IOC_WRITE, 0x64, 0x40, 0x1c), 0xbebdf474) = 0
>
> b) then, the DoMapToUser error happens but does not abort
>
> c) then all /dev/dri/card* are scanned
>
> open("/dev/dri/card0", O_RDWR)          = 4
> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f20)   = 0
> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f20)   = 0
> close(4)                                = 0
> open("/dev/dri/card1", O_RDWR)          = 4
> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f70)   = 0
> ioctl(4, DRM_IOCTL_VERSION, 0x4f0f70)   = 0
> close(4)                                = 0
> open("/dev/dri/card2", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card3", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card4", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card5", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card6", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card7", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card8", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card9", O_RDWR)          = -1 ENOENT (No such file or directory)
> open("/dev/dri/card10", O_RDWR)         = -1 ENOENT (No such file or directory)
> open("/dev/dri/card11", O_RDWR)         = -1 ENOENT (No such file or directory)
> open("/dev/dri/card12", O_RDWR)         = -1 ENOENT (No such file or directory)
> open("/dev/dri/card13", O_RDWR)         = -1 ENOENT (No such file or directory)
> open("/dev/dri/card14", O_RDWR)         = -1 ENOENT (No such file or directory)
> open("/dev/dri/card15", O_RDWR)         = -1 ENOENT (No such file or directory)
>
> vs. successful
>
> open("/dev/dri/card0", O_RDWR)          = 4
> ioctl(4, DRM_IOCTL_VERSION, 0x43df20)   = 0
> ioctl(4, DRM_IOCTL_VERSION, 0x43df20)   = 0

Right here is where it appears to change.  Any ideas what the
difference is between card0 on the different platforms?

> ioctl(4, DRM_IOCTL_SET_VERSION, 0xbebdfabc) = 0

Above it closed card0 and started to open card1 indicating to me it
didnt' find what it wanted.  Do you know what it wanted?

> ioctl(4, DRM_IOCTL_VERSION, 0x43df68)   = 0
> ioctl(4, DRM_IOCTL_VERSION, 0x43df68)   = 0
> open("/dev/dri/renderD128", O_RDWR|O_LARGEFILE) = 5

Looking at your logs (c), the above doesn't show it trying to open
renderD128, but the log in (a) seems to start with opening renderD128.
I am not very familiar with the 3D stuff, but
> ioctl(5, DRM_IOCTL_VERSION, 0x43df20)   = 0
> ioctl(5, DRM_IOCTL_VERSION, 0x43df20)   = 0
> ioctl(5, DRM_IOCTL_I810_FLUSH or DRM_IOCTL_MGA_SWAP or DRM_IOCTL_R128_CCE_RESET or DRM_IOCTL_RADEON_CP_RESET, 0) = -1 EACCES (Permission denied)
> fcntl64(5, F_SETFD, FD_CLOEXEC)         = 0
>
>
> 3. I have searched for the source code and things seem to fail in libdrm - which
> is good, because it is open source :)

I was investigating some issues with the Etnaviv driver on the i.MX6 a
while back, and there was an issue where the DMA driver changed the
available memory.  For while, I created a reserved memory pool in the
device tree which fixed the issue until a permanent solution was
found.  I haven't looked at the full device tree of the am33xx you're
using, but do you know if it's using any sort of reserved memory pool?
If there is a difference between the am33xx and dm3730, it might be
something to investigate.

>
> First of all calling DRM_IOCTL_VERSION is to get the string lengths by the first
> call to be able to allocate string buffers for the second call.
>
> The code looks sane and takes care about length and extra \0 for C strings.
>
> So next step will be to build my own version from sources and add some test code
> to understand why the search for the SGX fails. It would also be possible to use
> gdb gles1test1 and single-step through drmGetVersion() but that also needs source
> code...
>
> Anyways there are interesting parts in libdrm-2.4.74/xf86drm.c

Are you only trying the 3D in Xf86?

adam

>
> BR,
> Nikolaus
>
> _______________________________________________
> https://github.com/openpvrsgx-devgroup/linux_openpvrsgx
> openpvrsgx-devgroup mailing list
> openpvrsgx-devgroup at letux.org
> http://lists.goldelico.com/mailman/listinfo.cgi/openpvrsgx-devgroup


More information about the openpvrsgx-devgroup mailing list