Pain and swelling at night indicate inflammation. At this stage of an injury this shouldn't be happening. For this reason I would recommend you return to your Doctor. There may be more damage than just the fractures such as tendon or ligament damage that was not picked up or ignored because the fracture was thought worse. Soft tissue injuries are often more troublesome than a fracture.