Run hciserveradmin from the command line of your xterm session.
Then the login stuff should appear. Make sure your certs are in place.
I’ve had problems editing the server.ini directly before.
You may be able to get away with it.
It looks like you are having an xwindows problem.
See if you get the same error running tclhelp.
-mh
