Starting simple httpserver freezes Blender on quit

Originating in this topic: Multithreading support (please :) )
I wanted to have an external tool to trigger some buttons in blender, like a flight check, are normals ok? export as fbx with my settings, and other buttons I have to push all the time.

Having a simple httpserver sounded great.
It also works, kind of. But when I wrap it in an addon it will always freeze blender when I exit blender.
Any ideas how I could get this to work? I am out of trial and errors.

bl_info = {
    "name": "My Awesome Server",
    "blender": (2, 80, 0),
    "category": "Object",
}


from http.server import BaseHTTPRequestHandler, HTTPServer
import threading
import bpy


class MyServer(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == '/test':
            self.send_response(200)
            self.send_header('Content-type','text/html')
            self.end_headers()
            message = "This is a test page2."
            self.wfile.write(bytes(message, "utf8"))
        else:
            self.send_response(404)
            self.send_header('Content-type','text/html')
            self.end_headers()
            message = "404 Not Found."
            self.wfile.write(bytes(message, "utf8"))
        return


httpd = None
thread = None
thread2 = None


def start_server():
    global httpd
    
    server_address = ('localhost', 8080)
    # Server is not running, start it
    httpd = HTTPServer(server_address, MyServer)
    print("Server started")
    httpd.serve_forever()


def stop_server():
    global httpd
    global thread
    httpd.shutdown()
    thread.join()
    print("Server stopped")


def stop_server_thread():
    global thread2
    thread2 = threading.Thread(target=stop_server)
    thread2.start()


def register():
    global thread
    print("Hello World")

    # Start the server on a new thread
    thread = threading.Thread(target=start_server)
    thread.daemon = True
    thread.start()



def unregister():
    print("Goodbye World")

    stop_server_thread()
    global thread2
    thread2.join()

This only happens when I did open a website, e.g.: http://localhost:8080/test

If I just open blender, it will load the server fine, and if I quit, it will quit blender.

But once I open blender, then connect to the server via http://localhost:8080/test, it will freeze on quit, until I load the page again, then it will quit fine.

This is so wicked.

Maybe can use asyncio and aiohttp, it will work in Blender. Like BlenderKit did.