PR D135370 implemented a performance improvement but it restricted the filtering of declaration from inline namespace too much. In particular it did not filter for the function template case.
This led to a regression and this PR removes that check.
This fixes: https://github.com/llvm/llvm-project/issues/61851