“When you are a huge club, the demand is to win,” said Celtic manager Brendan Rodgers.
“In the last couple of years we’ve done that really well and playing in a style that is synonymous with the club. It hasn’t been the same but we have to work to make it better.”
Rodgers made six changes from the side that laboured to a 1-0 Europa League win over Rosenborg in midweek.
Leigh Griffiths was recalled to the starting line-up after coming off the bench to grab the winner against the Norwegian champions and his diving header put the visitors in front 11 minutes before half-time.
Celtic posted a record turnover of £100 million this week, but their failure to strengthen in the transfer window despite selling star striker Moussa Dembele to Lyon is already being exposed in their bid for an eighth straight Scottish title.
They have now scored just six times in their opening six games and rarely looked like adding to their lead before Chris Burke fired in off the post from 20 yards to bring Kilmarnock level.
Celtic continued to dominate possession without seriously threatening, and were caught again by a sucker punch in the 93rd minute when Findlay stooped to head Burke’s corner into the far corner, sparking wild scenes of celebration from Killie boss Steve Clarke.
In four meetings with Rodgers since Clarke took charge just under a year ago, Kilmarnock have now beaten and drawn with Celtic twice.
Rangers took full advantage to close to within five points of Hearts as they showed no hangover from an impressive 2-2 draw at Villarreal in the Europa League on Thursday.
Captain James Tavernier curled home a free-kick to open the scoring before Alfredo Morelos doubled the hosts’ advantage at Ibrox.
Scott Arfield, Kyle Lafferty and Daniel Candeias were also on target after the break, while Blair Alston scored a consolation for the Saints from the penalty spot.