One thing I noticed in the build output was the following:

Other Features:
  Inhibiting IPv6 canonicalization at startup.
  Compiling in support for dynamic shared object modules.
  Using the new portable dumper.
  Compiling in support for extra debugging code.
  Compiling in support for runtime error checking.
  WARNING: ---------------------------------------------------------
  WARNING: XEmacs will run noticeably more slowly as a result.
  WARNING: Error checking is on by default for XEmacs beta releases.
  WARNING: ---------------------------------------------------------

Boy, were they not kidding. I added --error-checking=none to the configure command inside carbon/build-app.sh, rebuilt, and my experience is noticeably zippier almost immediately.

New binary (47M) still with xemacs-sumo and xemacs-mule-sumo packages inside.