Bfile crash when writing G3A files specifically
» > >
»
» View previous topic :: View next topic  
In the process of writing a hex editor, I stumbled on a peculiar bug where writes to G3A files frequently fail with various System ERROR or reboots.

I originally thought this was a bug in my kernel, but upon closer inspection I can reproduce it easily on my fx-CG 50 with a simple vanilla/PrizmSDK program, shown below.

All it does is load Conv.g3a from the storage memory then rewrite it 100 times. When writing to a non-G3A file, eg. Conv.bin, it works no problem. But when writing to a G3A file, it always crashes in less than 10 runs. From other testing, I believe writing a G3A file that is obviously an invalid add-in (eg. file smaller than the G3A header) does not trigger the bug.

Archive with G3A file and buildable sources (up to toolchain name I believe): WriteG3A.zip

Obviously it must be related to add-in registration, but I don't know why that would be a problem for the OS.

Any ideas what might be causing the crash?

Code:
#include <fxcg/display.h>
#include <fxcg/keyboard.h>
#include <fxcg/file.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

char *conv_g3a;
#define SIZE 65536

int load(void)
{
    int fd = Bfile_OpenFile_OS(u"\\\\fls0\\Conv.g3a", BFILE_READ, 0);
    if(fd < 0) return -1;

    int rc = Bfile_ReadFile_OS(fd, conv_g3a, SIZE, -1);
    Bfile_CloseFile_OS(fd);
    return rc;
}

int save(int size)
{
    int fd = Bfile_OpenFile_OS(u"\\\\fls0\\Conv.g3a", BFILE_WRITE, 0);
    if(fd < 0) return -1;

    int rc = Bfile_WriteFile_OS(fd, conv_g3a, size);
    Bfile_CloseFile_OS(fd);
    return rc;
}

int main(void)
{
    conv_g3a = malloc(SIZE);
    if(!conv_g3a) return 1;

    int size = load();
    if(size < 0) return 1;

    char str[128];

    Bdisp_AllClr_VRAM();
    sprintf(str+2, "Loaded %d bytes", size);
    PrintXY(1, 1, str, 0, 0);
    Bdisp_PutDisp_DD();

    for(int i = 1; i <= 100; i++) {
        int rc = save(size);
        if(rc < 0) return 1;

        Bdisp_AllClr_VRAM();
        sprintf(str+2, "[%d] Saved %d bytes", i, rc);
        PrintXY(1, 1, str, 0, 0);
        Bdisp_PutDisp_DD();
    }

    return 1;
}
Apparently it's a known problem: https://www.casiopeia.net/forum/viewtopic.php?p=13684#p13684

With Yatis' help I tried a ton of variations. Basically if the program does writes to G3A files, regardless of whether they are new or not, already registered in the menu or not, valid G3A contents or not, and regardless of the opening mode, it crashes sooner or later (sometimes it takes 50 iterations but usually it crashes immediately).

Ah yes, something of note: there is always a slightly long process (couple of seconds) before the error. This processing might occur as a result of the error. But there is also a chance that this is simply the process of relocating data around in SMEM, and that it contributes to the error. (However, this happens even for writes to already-allocated files that should not require moving stuff around, and sometimes there are long delays that are not followed by crashes.)

Any insight appreciated as always.
There are many more details here: https://www.casiopeia.net/forum/viewtopic.php?p=13830#p13830

At the time they believed the problem was that the TLB would no longer point to add-in code after the file operation, resulting in obvious crashes when going back into the add-in. I'm not sure why this would happen, since Bfile_WriteFile() can (and could at the time) expand past the end of files, thus resizing dynamically, and still handle TLB updates for the running add-in if its data was moved in ROM. Maybe they didn't bother with this complicated process for add-in registration, which they assumed no add-in would ever trigger.

SimLo and gbl08ma hypothesized back then that running code from RAM could work. I have a prototype from KBD2 that performs SMEM optimization from code located in RAM then restarts the add-in. This is for the G-III. I might give that RAM method a try just to see, though I'm not expecting any breakthrough.
Right, last post for today.

Putting the file manipulation function in RAM, then leaving the add-in without executing any ROM code after the write succeeds. The code below does this; we have to be careful to copy syscalls like Bfile_CloseFile_OS() into RAM. Also we can't exit via exit(), so I use APP_MEMORY() as a fallback.

Code:
#include <fxcg/display.h>
#include <fxcg/keyboard.h>
#include <fxcg/file.h>
#include <fxcg/system.h>
#include <fxcg/app.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

char *conv_g3a;
#define SIZE 65536

int load(void)
{
    int fd = Bfile_OpenFile_OS(u"\\\\fls0\\Conv.g3a", BFILE_READ, 0);
    if(fd < 0) return -1;

    int rc = Bfile_ReadFile_OS(fd, conv_g3a, SIZE, -1);
    Bfile_CloseFile_OS(fd);
    return rc;
}

typedef unsigned uint;

typedef struct {
  uint VPN  :22;
  uint D    :1;
  uint V    :1;
  uint ASID :8;
} __attribute__((packed, aligned(4))) utlb_addr_t;

typedef struct {
  uint      :3;
  uint PPN  :19;
  uint      :1;
  uint V    :1;
  uint SZ1  :1;
  uint PR   :2;
  uint SZ2  :1;
  uint C    :1;
  uint D    :1;
  uint SH   :1;
  uint WT   :1;
} __attribute__((packed, aligned(4))) utlb_data_t;

#define utlb_addr(E) \
  ((utlb_addr_t *)(0xf6000000 | ((E & 0x3f) << 8)))

#define utlb_data(E) \
  ((utlb_data_t *)(0xf7000000 | ((E & 0x3f) << 8)))

void invalidate_rom_pages(void)
{
  for(int E = 0; E < 64; E++) {
    utlb_addr_t *addr = utlb_addr(E);
    utlb_data_t *data = utlb_data(E);
    if(!addr->V || !data->V) continue;

    uint32_t src = addr->VPN << 10;
    if(src >= 0x00300000 && src < 0x00700000) {
      data->V = 0;
      addr->V = 0;
    }
  }

  __asm__("icbi @%0":: "r"(0x00000000): "memory");
}

int (*_save)(int size) = NULL;
int (*_close)(int fd) = NULL;
void (*_app_memory)(void) = NULL;
void (*_invalidate)(void) = NULL;

int save(int size)
{
    uint16_t const *path = u"\\\\fls0\\Conv2.g3a";
    int rc = 0;

    rc = Bfile_DeleteEntry(path);

    size_t size2 = 0;
    rc = Bfile_CreateEntry_OS(path, BFILE_CREATEMODE_FILE, &size2);
    if(rc < 0) return -1;

    int fd = Bfile_OpenFile_OS(path, BFILE_READWRITE_SHARE, 0);
    if(fd < 0) return -1;

    // From now on the program will not return to mapped ROM, instead only
    // calling into code that lives in RAM or in the OS

    rc = Bfile_WriteFile_OS(fd, conv_g3a, size);
    (*_close)(fd);

    // Here we can try and invalidate the TLB manually. It definitely helps,
    // but it's not sufficient to allow the add-in to keep executing (the
    // errors are less severe though)
    /* (*_invalidate)();
       return rc; */

    // Leave the add-in without going back to ROM code
    (*_app_memory)();
    return rc;
}

int main(void)
{
    conv_g3a = malloc(SIZE);
    if(!conv_g3a) return 1;

    int size = load();
    if(size < 0) return 1;

    _save = malloc(2048);
    if(!_save) return 1;
    memcpy(_save, save, 2048);

    _close = malloc(128);
    if(!_close) return 1;
    memcpy(_close, Bfile_CloseFile_OS, 128);

    _app_memory = malloc(128);
    if(!_app_memory) return 1;
    memcpy(_app_memory, APP_MEMORY, 128);

    _invalidate = malloc(1024);
    if(!_invalidate) return 1;
    memcpy(_invalidate, invalidate_rom_pages, 1024);

    char str[128];

    Bdisp_AllClr_VRAM();
    sprintf(str+2, "Loaded %d bytes", size);
    PrintXY(1, 1, str, 0, 0);
    Bdisp_PutDisp_DD();

    for(int i = 1; i <= 100; i++) {
        int rc = (*_save)(size);
        if(rc < 0) return 1;

        Bdisp_AllClr_VRAM();
        sprintf(str+2, "[%d] Saved %d bytes", i, rc);
        PrintXY(1, 1, str, 0, 0);
        Bdisp_PutDisp_DD();
    }

    return 1;
}

Yatis suggested to invalidate the TLB manually so we can continue running the add-in instead of just jumping into another application. The code above contains the invalidate_rom_pages() function to do that, and we can in fact call (*_invalidate)() and then return. Unfortunately another error occurs there. It's more consistent and not as severe (the add-ins are no longer gone) so maybe a step in the right direction.
Good project, Lephe, I hope you will succeed. I also had similar problems a few years ago and if my recollection is correct the crashes would happen when renaming/moving add-ins which appear in the main menu before the actual add-in manipulating them - I believe the working solution at the end for me was to allow manipulation of the add-ins which appear after the add-in which manipulates them. Perhaps loading your add-in from some fixed location in memory could also make it immune to crashes which OS causes due to some memory remapping when main menu items get reshuffled. I had no time to reread the links you posted - just wanted to mention what I recall as I don’t remember if it was mentioned anywhere public.
Pokémon+/Lock Add-In: Prizm Password Protection
Message Add-In for Prizm/Language Add-In Template
G-PLUS & G-CLOCK: OS statusbar & G-SHOCK watch
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
Bfile crash when writing G3A files specifically
»
» View previous topic :: View next topic  
Page 1 of 1
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement