Last week, I decided to build a real-time application. I started the process by designing the architecture of the system, from the console to the ASYNC API to the backend engine powered by MySQL and Redis. If you're like me, you know I love decoupling my application as much as possible, so I started working on an async engine to provide an interface as a facade to my database models. Since I usually don't do concurrency in Python, this was my first experience with it. Python doesn't support concurrency out of the box, so developers need to really understand concurrency before trying to use it in Python.
I began by researching how to create an async engine using SQLAlchemy. I designed my models, translated them to code, and then exposed an engine so that all parts of my application could use the interface provided by the engine to communicate with the database.
Bug Alert: Houston, We have A problem
After a lot of designing and code writing, it was time to test the engine. I knew it wouldn't work immediately, so I was ready. I wrote my first test to import my engine, create an instance of my model, and then save a test user to the database using the interface exposed by my engine. That worked after some fixes, which gave me more confidence to add a test that would check if the same user I had created exists. If it existed, I would try creating the same user again and expect an error.
Instead of the integrity error that I was expecting, I got the "event loop closed" error. I laughed, took a deep breath, and knew it was time to debug
Byte by Byte: Piecing Together the Puzzle
Immediately after I ran the code, I got a traceback with the "event loop closed" error. I knew that wasn't supposed to happen since I was disposing off my SQLAlchemy database engine in the main function scope. I zoomed in on the error and found out my SQLAlchemy connection was being garbage collected, which wasn't expected since I was explicitly closing the connection after the error. I also knew that my session was supposed to return the same session every time it was called in the same async task. So, I knew something was wrong. I hopped into the documentation and read every description on how the connection pool works in SQLAlchemy. I also learned about the strategy for checking in and checking out of each connection from the connection pool.
Once I was done reading the documentation on the connection pool, I enabled the echo of pool check-in and check-out logs. This showed me that a connection wasn't being checked in after checking out and was getting garbage collected after I disposed off my engine at the end of the main function.
Squashing Bugs: The Victory Dance
At this point, I knew it would be trivial to catch the bug since I already knew its whereabouts. I ran another test to confirm that I was getting a new session each time and used the close_all function from SQLAlchemy to close any session, to prevent and confirm that the error wouldn't occur. From my test, I realized that I was indeed getting a new session from the session factory, which was supposed to return the same session object when called from the same async task. I inspected my code and discovered that I was initializing the session factory every time I called the session from my database engine. Hence, the factory was getting initialized every time, resulting in different sessions. Moving the initialization method of the session factory to my engine initialization fixed the error since I am using a singleton pattern for the engine.
Now I can use that confidence from bug fixing to tackle my upcoming HNG internship for the next 6 to 8 weeks. I am really pumped 💪.
The Road Ahead: My HNG Wishlist
Hands-on Experience: Dive deep into real-world projects and apply my skills in a practical setting.
Learning Opportunities: Gain new insights and knowledge, especially in areas I'm less familiar with.
Networking: Connect with like-minded individuals and industry professionals through HNG POOL.
Mentorship: Receive guidance and feedback from experienced mentors.
Skill Enhancement: Improve my problem-solving and debugging skills.
Collaboration: Work effectively within a team and contribute to collective goals.
Career Growth: Leverage this internship as a stepping stone for future opportunities.