Criticisms of Rio 2016 could be seen as a bid to talk up London 2012, claims BOA vice-chairman Hugh Robertson.
British athletes have been warned against voicing any criticisms of Rio 2016 for fear it could come across as arrogant.
Problems with transport, weather and empty venues have been among the issues affecting this year's Games, which got under way with a cut-price opening ceremony on Friday.
And Hugh Robertson - former UK sports and Olympics minister and current vice-chairman of the British Olympic Association - says talking down Rio's hosting of the event may be perceived as an attempt to blow London's trumpet, after the English capital hosted a hugely successful Games four years ago.
"I spoke to the team leaders at the last meeting before they left and I said: 'Just remember Brits do have a reputation for this touch of arrogance in the international sports world and it's really important that whatever the frustrations are we get behind this and are positive about it,'" he told reporters.
"It's enormously annoying when you get involved in some of the things that have been going on but it is really important for us that we don't come here and stick too many pins into it, particularly the local organising committee, because the suspicion will be that we are just doing this to big up London.
"They will be particularly sensitive to any criticism from us, so it's really important we don't get caught up in that."