SQLite is a good option for this.
I had a 70mb table that was way to big for .tbl or a .csv lookup.
Ended up going with SQLite which slowed down translation initially but then when I configured the value I was searching on as the primary key and with an index, 20 transactions processed at practically the same speed as what they did before the lookup was added.