Broken screen & digitizer, help with adb?

So I have an encrypted Nexus 5 loaded with TWRP custom recovery. Unfortunately, the screen and digitizer are broken. I see there is a tool to control my phone from my computer, but it requires usb debugging enabled. I also see I can enable usb debugging from adb, and I can see just barely enough to get into recovery mode from the bootloader. However, I am unable to see or enter the pin for the screen pattern from TWRP (because of the encryption). I do have access with fastboot at the bootloader. Am I out of luck or is there any way to proceed at this stage?