Ambiguous answers
Depending on the type of question asked, when asked natural questions that force the application to give an opinions, some of the applications supply tentative answers – rather than take a hard and fast opinions like a human author, they often sit on the fence, with their answers being
Recency/Niche topics
Quality of answer will depend on how much has been published on the area – if it’s a recent topic, which not much published on it yet, or a really niche topic with little interest, then your answer may not be very trustworthy.
Some of the applications are simply better than the others. Some may mistakes in not finding articles, or providing the wrong author in an author search.
Quality/Size of sources
It's also important to check what database the articles are sourced from. Some will be sourced from larger databases, some from smaller. Some will be sourced only from open source, whereas other will be sourced from proprietary subscription databases.
Access to Full Text
Unlike commercial subscription journal searches, Gen AI Lit review software is often not supported or connected to large databases of journal articles.
While Semantic Scholar has made deals and contacted various publishers, others have not. Often when looking through search results, the user will be presented with links to sign into a subscription to read the full text, which they may not have.