Scratch-3D

/assets/images/Scratch/scratch-capture-1.png

Description

Scratch-3D is a 3D rendering engine I’ve been working on as a pet project.

I built it to have a playground for shader effects and any other graphics things I wanted to play with. It has a fully featured scene editor that allows for quick positioning and editing of shader properties. It can also serialize and load scenes to and from a json file.

I continue to tinker with it whenever I have spare time. I’m hoping to add Vulkan and/or DirectX support in the future as well as more editor features and potentially a runtime environment (If I need it).

Gif of Engine

The code is fully open-source and available at https://github.com/jaideng123/Scratch

About the Project

Made with: C++, OpenGL, ImGui

Role: Sole Developer

Project Length: 1 year (on and off)

Resources I used:

My main motivation for starting this project was to learn a bit more about how rendering engines work under the hood and de-rust my C++ skills(since I mostly work in Unity).

I started by going through most of the tutorials on learnopengl.com to get a solid grasp on how it’s rendering API worked. Once I had that that knowledge as a base I researched different rendering engine architectures to figure out what I wanted to go for. There’s quite a bit of methods out there (which was a little overwhelming) but I settled on Ogre3D as my primary inspiration since it was fairly mature, had good documentation and was fully open source.

I started hacking away on nights and weekends getting the basic structure together and rendering a simple scene with the nano-suit model. I quickly realized there was a lot of work that went into fully-featured 3D engines so I had to pick and choose carefully what features I would implement, find a library for, or just not do entirely to keep the project from ballooning out of control. A good example of this is how I opted not to use a custom memory manager and relied mainly on standard library smart pointers since the performance they provided was good enough for my purposes and it greatly simplified my memory management.

Once I had the basic scene working, I started adding editor features to make life easier for myself. I used ImGui as my UI library because it came highly recommended in custom-engine circles and I found the API to be very comfortable coming from a React background. I used ImGuizmo to quickly add a gizmo and transform editor to my scene and manipulate the entities I had created. One feature I’m particularly proud of is the pixel perfect mouse selection that uses a custom shader to render an object’s ID to a buffer and cross-reference that with mouse position.

In addition to the new UI features I also added serialization for the entire scene. I chose JSON as my file format because it would be easy to debug and I had a lot of experience with it(even though it can be a little slow). Serialization and Deserialization changed a fair amount of my architecture and forced me to reckon with some unfortunate architecture decisions I made early on, but after some re-jiggering I was able to get it working with minimal hacks.

As of writing, this project has taken quite a bit of time and effort so I’m shelving it for the time being, but I will likely tinker with it some more when I get more time and find something I really want to implement using it.

Update 01/11/2022

Since I had some free time over the holiday, I decided to come back to this project and fix some of the things that were bothering me about the overall structure.

I started by mapping out the current structure of the engine along with the main goals I had in mind for the refactor, those being:

The ability to cache resources (model data, textures, materials, etc.)
A simpler, more flexible rendering API, that could support culling in the future
Material instancing so that all entities didn’t have to share materials

To accomplish goal number 1, I created a resource manager that would hold all the various resources in memory and when called for a resource, check if it had been loaded into memory before loading it. This madee all my loading MUCH more efficient, loading 100 of the same model used to take a few minutes and now takes seconds. It also had the knock-on effect of making my structure much cleaner by defining a clear owner for these resources. In the future this resource manager could also evict unused resources to free up memory, but right now the scale of my scenes don’t really necessitate it.

For goal number 2, I looked to inspiration from Unity, as they have a pretty clean, simple interface for submitting things to be rendered. Originally, I had hacked in the transform matrix as a part of the material, but now looking back in hindsight this was VERY BAD, caused a lot of problems, and just didnt make a lot of sense. In the new API, you just need to submit a mesh, a material, and a transform, and we store them all separately until it’s time to render, much cleaner and much simpler. Another thing I wanted to simplify was the way renders were triggered, before I had a very clumsy process that had the scene gathering up all the meshes, but now that we could just arbitrarily submit them and let the render system hold onto them for the frame, a render call just needs a camera, and a directional light. In the future I’m planning to use the camera to do frustrum culling, add a depth prepass, and sort the render queue by material to squeeze more performance out of the overall system.

The last goal didn’t work out as well as I’d hoped, the solution I ended up going with forced each entity to contain a material instance (copied from a default material set when a model is first loaded). This allowed for each object in a scene to have separate material properties but also meant I could no longer batch materials together during renders. I’ll have to do some more research and thinking about a better system I could use here, because I’m definitely not happy with the curent system.

I also took this opportunity to address some low-hanging fruit like the ability to set shaders on each material, store material properties as binary data instead of strings (which led to a 2x perf increase), pulling Scene out into a proper object, and cleaning up a lot of my TODOs that I left behind.