Liquid Labs LLC v. FDA ( 2022 )


Menu:
  •                                       PRECEDENTIAL
    UNITED STATES COURT OF APPEALS
    FOR THE THIRD CIRCUIT
    ______________
    No. 21-2883
    ______________
    LIQUID LABS LLC,
    Petitioner
    v.
    UNITED STATES FOOD AND DRUG
    ADMINISTRATION
    ______________
    On Petition for Review of a Final Marketing Denial Order
    By the United States Food and Drug Administration
    (Agency Nos. PM0003412 & PM0000984)
    ______________
    Submitted Under Third Circuit L.A.R. 34.1(a)
    October 3, 2022
    ______________
    Before: CHAGARES, Chief Judge, SHWARTZ and
    SCIRICA, Circuit Judges.
    (Filed: October 27, 2022)
    Kevin T. Duffy, Jr.
    Duffy & Staab
    165 West Putnam Avenue
    Suite 2B
    Greenwich, CT 06903
    Gary G. Staab
    Duffy & Staab
    2 William Street
    The Kennedy Building
    Suite 302
    White Plains, NY 10601
    Counsel for Petitioner
    Arun G. Rao
    Gustav W. Eyler
    Hilary K. Perkins
    Jonathan E. Amgott
    United States Department of Justice
    Consumer Protection Branch
    450 5th Street, N.W.
    Suite 6400 South
    Washington, DC 20001
    Brian M. Boynton
    Joshua Koppel
    United States Department of Justice
    Civil Division
    950 Pennsylvania Avenue, N.W.
    Room 7212
    Washington, DC 20530
    Daniel J. Barry
    United States Department of Health & Human Services
    200 Independence Avenue, S.W.
    2
    Washington, DC 20201
    Wendy S. Vicente
    Marci B. Norton
    Food and Drug Administration
    Office of the Chief Counsel
    10903 New Hampshire Ave.
    Silver Spring, MD 20993
    Counsel for Respondent
    James G. Troutman
    Troutman Law Office
    4205 Springhurst Boulevard
    Suite 201
    Louisville, KY 40241
    Counsel for Amicus Petitioners 38 National and
    State Electronic Nicotine Delivery System Product
    Advocacy Associations
    Mary G. Bielaska
    Zanicorn Legal
    845 Third Avenue
    6th Floor
    New York, NY 10022
    Counsel for Amicus Petitioners Dr. David B.
    Abrams,Clive D. Bates, and Professor David T.
    Sweanor, J.D.
    William B. Schultz
    Zuckerman Spaeder
    1800 M Street, N.W.
    Suite 1000
    Washington, DC 20036
    3
    Counsel for Amicus Respondents Medical and
    Public Health Groups
    ______________
    OPINION OF THE COURT
    ______________
    SHWARTZ, Circuit Judge.
    Liquid Labs LLC (“Liquid Labs”) sought permission
    from the Food and Drug Administration (“FDA”) to market
    products used in e-cigarettes. The FDA denied the request, and
    Liquid Labs petitions for review. Because the FDA’s order
    was within its statutory authorities and the Administrative
    Procedure Act (“APA”), we will deny the petition.
    I
    A
    E-cigarettes are electronic nicotine delivery systems
    (“ENDS”) that vaporize e-liquids and allow for inhalation. 1
    1
    Some e-cigarettes are disposable, while others are
    reusable. Within the reusable group, some e-cigarettes have
    “open systems,” meaning they are “refillable” and “include[] a
    reservoir that a user can refill with an e-liquid of their
    choosing,” JA 210; some have “closed systems,” meaning, for
    example, they “use[] e-liquid contained in replaceable
    cartridges or pods that are not intended to be refillable,” JA 210
    4
    See, e.g., Big Time Vapes, Inc. v. FDA, 
    963 F.3d 436
    , 439 n.11
    (5th Cir. 2020), cert. denied, 
    141 S. Ct. 2746
     (2021). Liquid
    Labs manufactures and sells e-liquids that generally contain
    nicotine and flavoring.
    Liquid Labs’ e-liquids qualify as “new tobacco
    product[s]” under the Family Smoking Prevention and
    Tobacco Control Act (the “Act”). See 
    21 U.S.C. §§ 387
    -387u.
    The Act applies to “all cigarettes, cigarette tobacco, roll-your-
    own tobacco, and smokeless tobacco and to any other tobacco
    products that the Secretary by regulation deems to be subject
    to” the Act. 21 U.S.C. § 387a(b); see also 21 U.S.C.
    § 387j(a)(1)(A) (defining “a new tobacco product,” as relevant
    here, to be “any tobacco product . . . that was not commercially
    marketed in the United States as of February 15, 2007”). In
    2016, the FDA “deem[ed]” e-cigarettes and related
    components (such as Liquid Labs’ e-liquids) to be subject to
    the Act’s requirements. See Deeming Tobacco Products To Be
    Subject to the Federal Food, Drug, and Cosmetic Act, 
    81 Fed. Reg. 28,974
    , 29,028 (May 10, 2016); see also Big Time Vapes,
    963 F.3d at 440.
    Because Liquid Labs’ e-liquids qualify as new tobacco
    products, they may not be introduced into interstate commerce
    without the FDA’s authorization. See 21 U.S.C. § 387j(a)(2).
    One way to obtain authorization is by submitting a premarket
    tobacco product application (“PMTA”). See, e.g., Big Time
    Vapes, 963 F.3d at 439; see also 21 U.S.C. § 387j(b)-(c).
    and some have “mod[ifiable] system[s]” that allow the user to
    adjust various aspects of the e-cigarette, see, e.g., JA 135.
    Liquid Labs’ e-liquids are used in connection with open
    systems.
    5
    Under the Act, the FDA “shall deny” a PMTA if the
    applicant fails to “show[] that permitting such tobacco product
    to be marketed would be appropriate for the protection of
    public health.” 21 U.S.C. § 387j(c)(2)(A). “[T]he finding as
    to whether the marketing of a tobacco product . . . is
    appropriate for the protection of the public health [is]
    determined with respect to the risks and benefits to the
    population as a whole, including users and nonusers of the
    tobacco product.” 21 U.S.C. § 387j(c)(4). On this subject, the
    Act directs the FDA to “tak[e] into account” both “the
    increased or decreased likelihood that existing users of tobacco
    products will stop using such products,” and “the increased or
    decreased likelihood that those who do not use tobacco
    products will start using such products.”            21 U.S.C.
    § 387j(c)(4)(A)-(B).
    In addition to the Act and the deeming regulation, the
    FDA took several related regulatory steps. For example, the
    FDA issued guidance in June 2019 (“June 2019 Guidance”)
    and April 2020 (“April 2020 Guidance”) that “help[ed]
    manufacturers prepare [PMTA] applications ahead of the
    [discretionarily delayed submission] deadline,” and “set[] out
    the agency’s enforcement priorities,” respectively. Prohibition
    Juice Co. v. U.S. Food & Drug Admin., 
    45 F.4th 8
    , 13-15 (D.C.
    Cir. 2022). Among other things, these documents highlighted
    that flavored e-liquids’ had a “disproportionate appeal to
    children,” id. at 13, and “noted the types of rigorous scientific
    evidence [the FDA] would accept in support of applications to
    market such products,” id. at 15.
    6
    B
    Liquid Labs submitted two PMTAs on September 4,
    2020, covering twenty e-liquid products. The products
    spanned ten flavors, two of which are described as being
    tobacco flavored, and eighteen of which are described as
    having a “characterizing flavor” other than tobacco or menthol
    with names such as “OG Island Fusion,” “Berry Au Lait, “OG
    Summer Blue,” and “Shake.”
    In connection with the applications, Liquid Labs
    submitted evidence from a variety of sources, including an
    abuse liability study, a cross-sectional perception and intention
    study, a population modeling analysis, a clinical literature
    review, and “well-controlled non-clinical analyses of Liquid
    Labs’ Products.” Pet. Br. at 20. Liquid Labs also submitted a
    marketing plan setting forth, among other things, various
    measures Liquid Labs planned to take to discourage youths
    from using its products.
    In September 2021, the FDA denied Liquid Labs’
    PMTAs. 2 In connection with its denials, the FDA sent Liquid
    Labs several documents, including a Marketing Denial Order,
    a document titled “Technical Project Lead (TPL) Review of
    PMTAs,” JA 62, and two documents titled “Review for
    Flavored ENDS PMTAs,” JA 52, 57.
    2
    The denial order did not list Liquid Labs’ tobacco-
    flavored e-liquids as products “lack[ing] sufficient evidence to
    demonstrate that the marketing of th[e] products is appropriate
    for the protection of public health.” JA 10. Liquid Labs
    represents that approval for its tobacco-flavored e-liquids has
    been neither granted nor denied.
    7
    The Marketing Denial Order briefly explained why the
    applications “lack[ed] sufficient evidence to demonstrate that
    the marketing of the[] products [wa]s appropriate for the
    protection of public health.” JA 1. It noted, for example, that
    “[i]n light of the known risks to youth of marketing flavored
    ENDS, robust and reliable evidence” was “needed regarding
    the magnitude of the potential benefit to adult smokers,” and
    such evidence could have been provided through “randomized
    controlled trial[s] and/or longitudinal cohort stud[ies],” as well
    as through “other evidence[,] but only if it reliably and robustly
    evaluated the impact of the new flavored vs. tobacco-flavored
    products on adult smokers’ switching or cigarette reduction
    over time.” JA 1. The FDA found that Liquid Labs’ cross-
    sectional survey was “not sufficient” because “it d[id] not
    evaluate product switching or cigarette reduction resulting
    from use of these products over time or evaluate these
    outcomes based on flavor type to enable comparisons between
    tobacco and other flavors.” JA 1-2.
    The Technical Project Lead Review further discussed
    the FDA’s rationale for denying Liquid Labs’ applications.
    For example, it set forth the FDA’s concern about youth use of
    flavored ENDS and regulatory actions the FDA has taken to
    address the issue. It also explained, among other things, (1)
    why the FDA focused “on the risk to youth nonusers as well as
    the potential benefit to adult smokers as current users,” (2) why
    “only the strongest types of evidence” would be sufficient to
    show an adequate benefit to adult smokers, (3) why the FDA
    looked for “acceptably strong evidence that the flavored
    products have an added benefit relative to that of tobacco-
    flavored ENDS in facilitating smokers completely switching
    away from or significantly reducing their smoking,” and (4)
    8
    how it concluded that although Liquid Labs’ applications
    “contain other evidence regarding the potential benefit to adult
    users,” the “other evidence [wa]s not adequate.” JA 64.
    The Technical Project Lead Review explained that
    Liquid Labs’ “internet-based cross-sectional survey” evidence
    “[wa]s not sufficiently strong to support the benefit to adult
    smokers of using these flavored ENDS because it d[id] not
    evaluate product switching or cigarette reduction resulting
    from use of these products over time or evaluate these
    outcomes based on flavor type to enable comparisons between
    tobacco and other flavors.” JA 75. Accordingly, the FDA
    concluded that Liquid Labs had not shown that the benefits of
    the products sufficiently outweighed the risks they posed to
    youths.
    In the Reviews for Flavored ENDS PMTAs, the FDA
    examined Liquid Labs’ submissions to see if they “contain[ed]
    evidence from a randomized controlled trial, longitudinal
    cohort study, and/or other evidence regarding the impact of the
    new ENDS on switching or cigarette reduction that could
    potentially demonstrate the benefit of [its] flavored ENDS over
    an appropriate comparator tobacco-flavored ENDS.” JA 53,
    58. The reviews noted that the PMTAs lacked both
    randomized controlled trials related to new product use and
    smoking behavior and longitudinal cohort studies on new
    product use and smoking behavior and one review specified
    that the “[o]ther evidence” submitted was “not sufficient to
    support the benefit to adult smokers of using these flavored
    ENDS . . . .” JA 54; see also JA 57 (noting “[e]vidence is
    absent in PMTAs”).
    9
    Liquid Labs petitions for review. 3
    II 4
    A5
    3
    Liquid Labs also sought re-review from the FDA.
    After completing its re-review, the FDA again concluded that
    Liquid Labs’ evidence did not “demonstrate a sufficient
    potential benefit to adult smokers to warrant rescission” of its
    prior denial. JA 317.
    4
    This Court has jurisdiction pursuant to 21 U.S.C.
    § 387l(a)(1)(B).
    5
    We review the FDA’s order denying the PMTAs under
    the standards set forth in the APA, see 21 U.S.C. § 387l(b)
    (noting a “regulation or denial” “shall be reviewed in
    accordance with section 706(2)(A)” of the APA), and, thus,
    may hold it “unlawful and set [it] aside” if “arbitrary,
    capricious, an abuse of discretion, or otherwise not in
    accordance with law,” 
    5 U.S.C. § 706
    (2)(A). An agency acts
    arbitrarily or capriciously if, for example, it “entirely fail[s] to
    consider an important aspect of the problem,” Motor Vehicle
    Mfrs. Ass’n of U.S., Inc. v. State Farm Mut. Auto. Ins. Co.,
    
    463 U.S. 29
    , 43 (1983), “offer[s] only a ‘conclusory statement’
    which ‘fail[s] to articulate a rational basis for its conclusion,’”
    or “cit[es] no data whatsoever in support of its decision,” Sierra
    Club v. U.S. Env’t Prot. Agency, 
    972 F.3d 290
    , 298 (3d Cir.
    2020) (first quoting W.R. Grace & Co. v. E.P.A., 
    261 F.3d 330
    ,
    342 (3d Cir. 2001), then quoting Natural Res. Def. Council,
    Inc. v. E.P.A., 
    790 F.2d 289
    , 309 (3d Cir. 1986)).
    When conducting this analysis, we “review the whole
    record or those parts of it cited by a party, and [give] due
    account . . . [to] the rule of prejudicial error.” 
    5 U.S.C. § 706
    ;
    10
    Liquid Labs contends that the FDA acted arbitrarily and
    capriciously in a number of respects. For the reasons below,
    we reject each of Liquid Labs’ arguments.
    1
    Liquid Labs first argues that the FDA acted arbitrarily
    and capriciously by “pull[ing] a surprise switcheroo” by
    “requir[ing]” certain evidence it previously indicated would
    not be necessary and rejecting evidence it led Liquid Labs to
    believe would be sufficient. 6 Pet. Br. at 39. In doing so, the
    see also Nat’l Ass’n of Home Builders v. Defs. of Wildlife, 
    551 U.S. 644
    , 659 (2007); Shinseki v. Sanders, 
    556 U.S. 396
    , 409
    (2009) (explaining that the “burden of showing that an error is
    harmful normally falls upon the party attacking the agency’s
    determination”). Concomitantly, we “judge the agency’s
    decision ‘solely by the grounds [it] invoked.’” Rad v. Att’y
    Gen., 
    983 F.3d 651
    , 656 (3d Cir. 2020) (quoting SEC v.
    Chenery Corp., 
    332 U.S. 194
    , 196 (1947)) (alteration in
    original).
    To the extent the issue pertains to an agency’s
    interpretation of the statutes it administers, we follow the
    “familiar Chevron framework,” first “giv[ing] effect to
    Congress’ unambiguous intent” “if the statute is clear,” and,
    second, “defer[ring] to an implementing agency’s reasonable
    interpretation of that statute” “if the statute is silent or
    ambiguous with respect to a specific issue.” Contreras Aybar
    v. Sec’y U.S. Dep’t of Homeland Sec., 
    916 F.3d 270
    , 273 (3d
    Cir. 2019) (quoting De Leon-Ochoa v. Att’y Gen., 
    622 F.3d 341
    , 348 (3d Cir. 2010)).
    6
    Specifically, Liquid Labs claims that the FDA
    unexpectedly required it to provide (1) randomized controlled
    11
    FDA, according to Liquid Labs, provided inadequate notice,
    upset its reliance expectations, and acted arbitrarily and
    capriciously.
    Liquid Labs relies on several documents the FDA
    issued between June of 2019 and September 2021. The first is
    the FDA’s June 2019 Guidance “intended to assist persons
    submitting . . . PMTAs.” JA 205. The June 2019 Guidance
    explained, among other things, that although the Food, Drug,
    and Cosmetic Act “states that the finding of whether permitting
    the marketing of a product would be [appropriate for the
    protection of public health] will be determined, when
    appropriate, on the basis of well-controlled investigations,” the
    FDA was also permitted to consider “other ‘valid scientific
    evidence’ if found sufficient to evaluate the tobacco product.”
    JA 216; see also JA 250 (“FDA believes that in some cases, it
    may be possible to support a marketing order for an ENDS
    product without conducting new nonclinical or clinical studies.
    . . . In cases where a product has not yet been sufficiently
    reviewed, new nonclinical and clinical studies may be
    necessary to support a marketing order.”). It further stated that
    “[n]onclinical studies alone are generally not sufficient to
    trials and/or longitudinal cohort studies, (2) studies occurring
    “over time” and/or “long-term studies,” and (3) studies
    comparing the “efficacy between different ENDS products.”
    Reply Br. at 5, 7, 10.
    Liquid Labs adds new arguments in its reply brief
    regarding a “net benefit” requirement and the FDA’s “refusal
    to request additional evidence before issuing denial.” See
    Reply at 12, 14. We decline to “reach arguments raised for the
    first time in a reply brief.” Barna v. Bd. of Sch. Dirs. of Panther
    Valley Sch. Dist., 
    877 F.3d 136
    , 146 (3d Cir. 2017).
    12
    support a determination that permitting the marketing of a
    tobacco product would be appropriate for the protection of
    public health.” JA 216. “Nonetheless,” the document
    continued, the FDA “in general,” did not “expect that
    applicants w[ould] need to conduct long-term studies to
    support an application.” JA 217. The Guidance also
    “recommend[ed that] an applicant compare the health risk of
    its product to both products within the same category and
    subcategory, as well as products in different categories as
    appropriate.” JA 217.
    According to Liquid Labs, the June 2019 guidance
    “encourag[ed] submission of the very evidence FDA
    [ultimately] reject[ed],” and “induced” the shortcomings
    highlighted in the FDA’s denial of Liquid Labs’ PMTA. Pet.
    Br. at 38. The second item that Liquid Labs relies on is a July
    2021 internal FDA memorandum, which explained, among
    other things, that the “[t]he absence of” “a randomized
    controlled trial” and/or “a longitudinal cohort study”
    constituted “a fatal flaw, meaning any application lacking
    [such] evidence w[ould] likely receive a marketing denial
    order,” JA 273-74, and an August 2021 internal memorandum
    that Liquid Labs asserts “justified” the July 2021
    memorandum, Pet. Br. at 26. 7
    7
    Liquid Labs relies heavily on the FDA’s July
    statements and its August justification that the absence of
    certain studies would constitute a “fatal flaw” in a PMTA
    application, but this ignores the fact that the memoranda that
    made this comment were rescinded either expressly or
    implicitly.
    13
    We join our sister circuit courts who have rejected these
    “surprise switcheroo” arguments. See Prohibition Juice, 45
    F.4th at 20-21; Wages & White Lion Invs., LLC v. FDA, 
    41 F.4th 427
    , 438-39 (5th Cir. 2022) 8; Breeze Smoke, LLC v. U.S.
    Food & Drug Admin., 
    18 F.4th 499
    , 506-07 (6th Cir. 2021),
    cert. denied, 
    142 S. Ct. 638
     (2021).
    With respect to the claim that the FDA surprisingly
    required randomized controlled trials and/or longitudinal
    cohort studies, “[t]he text of the FDA’s [June] 2019 Guidance
    makes . . . clear” that “the FDA did not reverse course.”
    Prohibition Juice, 45 F.4th at 21. Put simply, the FDA did not
    newly require those specific types of studies but instead found
    that Liquid Labs’ other evidence was inadequate. The 2019
    Guidance “said that [such studies] would not be necessary if
    applicants submitted similarly rigorous ‘valid scientific
    evidence,’” but “nowhere guaranteed that unspecified other
    forms of evidence would necessarily be sufficient—only that
    they might be, so the FDA would consider them.” Id.; accord
    Wages & White Lion, 41 F.4th at 438-39; Breeze Smoke, 18
    F.4th at 506-07; Gripum, LLC v. United States Food & Drug
    Admin., 
    47 F.4th 553
    , 559-60 (7th Cir. 2022). Further, the July
    2021 memorandum “did not necessarily foreclose reliance on
    other forms of rigorous evidence,” and the August
    memorandum “expressly required the agency to consider other
    8
    Liquid Labs relies heavily on an earlier opinion from
    the Court of Appeals for the Fifth Circuit granting a stay
    pending merits review. See Wages & White Lion Invs., LLC
    v. U.S. Food & Drug Admin., 
    16 F.4th 1130
     (5th Cir. 2021).
    The merits panel, however, denied the petition for review. See
    Wages & White Lion Invs., 
    41 F.4th 427
    .
    14
    forms of evidence if sufficiently robust.” Prohibition Juice, 45
    F.4th at 22.
    Further, in denying Liquid Labs’ applications, the FDA
    acted in conformity with the June 2019 Guidance. Nothing in
    the Marketing Denial Order, the Technical Project Lead
    Review, and the two Reviews for Flavored ENDS PMTAs
    “required” Liquid Labs to include “product-specific
    [randomized controlled trials]/longitudinal cohort studies.”
    Reply Br. at 5. 9 Each document states that the FDA would—
    and indicates that it in fact did—consider other evidence.
    Liquid Labs’ studies, however, did not produce the kind of
    evidence the FDA consistently sought. For example, the June
    2019 Guidance recommends that “PMTAs for flavored
    products [ ] examine [both] the impact of the flavoring on
    consumer perception . . . especially given the attractiveness of
    flavors to youth and young adults,” and the “adult appeal of
    9
    Liquid Labs also contends that an FDA press release
    “announced that it [would] require[] the very studies it
    originally expected it didn’t need.” Pet. Br. at 39. The press
    release does not reveal that the FDA changed its requirements.
    Indeed, it specifically states that “the agency [was] not
    foreclos[ing] the possibility that other types of evidence”—i.e.,
    other than randomized controlled trials and longitudinal cohort
    studies—“could be adequate if sufficiently robust and
    reliable.”   See Press Release, FDA Denies Marketing
    Applications for About 55,000 Flavored E-Cigarette Products
    for Failing to Provide Evidence They Appropriately Protect
    Public Health (Aug. 26, 2021), https://www.fda.gov/news-
    events/press-announcements/fda-denies-marketing-
    applications-about-55000-flavored-e-cigarette-products-
    failing-provide-evidence .
    15
    such flavors in their decisions to initiate use, cease use of more
    harmful products, or dual use.” JA 246; see JA 241-42
    (recommending “considering” “[p]ublished literature or
    applicant-initiated studies evaluating the effects of the ENDS
    on users, including effects on initiation, switching behavior,
    cessation, and dual use; and on nonusers’ initiation of the
    product.”). Further, the Guidance recommends that “an
    applicant compare the health risks of its products to both
    products within the same category and subcategory, as well as
    products in different categories as appropriate.” JA 217; see
    also JA 244. Liquid Labs’ abuse liability study compares its
    “OG Blue” flavor e-liquid with cigarettes and nicotine gum but
    not with its “Bacco” flavor or other tobacco-flavored e-liquid.
    Similarly, the cross-sectional survey neither shows a benefit to
    flavoring nor provides meaningful information regarding
    actual switching or reduction, and both Liquid Labs’ literature
    review and a third-party literature review indicate uncertainty
    regarding the role of flavors in smoking cessation. Thus, the
    FDA did not deny Liquid Labs’ applications solely because
    they lacked randomized controlled trials or longitudinal cohort
    studies. Rather, the record indicates that the FDA properly
    denied them because the other evidence Liquid Labs submitted
    was insufficient.
    Accordingly, the FDA did not “reverse course” and
    newly require randomized controlled trials and/or longitudinal
    cohort studies, and therefore did not upset Liquid Labs’
    reliance interests, provide inadequate notice, or act arbitrarily
    and capriciously.
    Contrary to Liquid Labs’ assertion, the FDA also did
    not arbitrarily and capriciously mandate “over time/long-term
    16
    studies.” Reply at 7. 10 The Marketing Denial Order, Reviews
    for Flavored ENDS PMTAs, and Technical Project Lead
    Review all demonstrate that the FDA wanted reliable evidence
    that Liquid Labs’ flavored e-liquids, among other things,
    helped adult smokers cut down on their cigarette use or switch
    to using ENDS products only. So did the FDA’s June 2019
    Guidance. See, e.g., JA 217. Reliable evidence of these
    behavioral changes is more likely to come from a study
    conducted over time because data collected over a short period
    may not show whether a particular change is temporary or
    long-lasting. To this end, the FDA stated that “it might accept
    evidence other than long-term studies, if that evidence had
    sufficient scientific underpinnings to meet the [Act]’s statutory
    mandate of demonstrating that flavored ENDS devices are
    appropriate for the protection of public health.” Breeze
    Smoke, 18 F.4th at 506-07 (emphasis omitted). Thus, the FDA
    does not require PMTA applicants to conduct long-term
    studies. 11
    10
    Liquid Labs conflates “long-term” studies with
    studies examining behavior “over time.” First, the FDA
    materials show that the phrase “long-term study” measures the
    duration of a study. For example, the FDA describes “long-
    term studies” as lasting six months or more. Studies that
    measure behavior over time, however, can last shorter periods.
    In fact, the Technical Project Lead Review indicates that
    studies occurring “over time” could be shorter than six months.
    Second, although a study concerning behavioral changes over
    time could be the focus of a “long-term” study, such a study
    may not be necessary to secure the information sought.
    11
    That said, to the extent Liquid Labs is claiming “the
    FDA’s statement that it would consider evidence other than
    long-term studies” “announc[ed]” that “‘long-term studies
    17
    Moreover, the FDA did not require “long-term studies”
    from Liquid Labs. Both the Marketing Denial Order and the
    Technical Project Lead Review refer to studies analyzing
    behavior “over time,” JA 1-2, 73-74 & n.xxiii, but, as
    explained above, that does not mean that a long-term study is
    required. Rather, “the FDA has all along required ‘valid
    scientific evidence,’ and its denial orders explained how the
    . . . data petitioners submitted fell short of the mark.”
    Prohibition Juice, 45 F.4th at 23. Accordingly, Liquid Labs’
    “over time/long-term studies” argument is also unavailing.
    2
    We also join our sister circuits in concluding that the
    FDA permissibly “required a comparison of a manufacturer’s
    ‘flavored products’ with ‘tobacco-flavored ENDS’ products in
    their ability . . . to assist adult smokers to quit or switch.” Reply
    Br. at 10 (emphasis omitted); see Wages & White Lion, 41
    F.4th at 434; Prohibition Juice, 45 F.4th at 19-20, 23-24. “The
    governing statute expressly asks for evidence concerning
    whether an applicant’s ‘tobacco product presents less risk than
    other tobacco products,’ . . . and the FDA’s [June] 2019
    were likely unnecessary,’” they are “over-read[ing]” it.
    Prohibition Juice, 45 F.4th at 22-23 (quoting Wages & White
    Lion, 16 F.4th at 1140-41). In its June 2019 Guidance, the
    FDA “broadened the types of evidence it would consider,”
    meaning “[i]nstead of limiting applicants to the two types of
    evidence it usually requires, the agency allowed manufacturers
    to submit evidence in other forms.” Id. at 21. “But at the same
    time the agency made clear it would not relax the scientific
    rigor of the requisite public health demonstration.” Id.
    18
    Guidance told manufacturers that the agency would look for
    comparisons between the proposed product and ‘tobacco
    products in the same category or subcategory.’” Prohibition
    Juice, 45 F.4th at 23 (first quoting 21 U.S.C. § 387j(b)(1)(A),
    then quoting June 2019 Guidance at 13). The “FDA is then
    required to consider ‘the information submitted to the
    Secretary as part of the application,’ which necessarily
    includes the comparative efficacy reports that applicants must
    provide.” Wages & White Lion Invs., 41 F.4th at 434 (quoting
    21 U.S.C. § 387j(c)(2)) (emphasis omitted). The FDA is also
    required to “consider ‘the increased or decreased likelihood
    that existing users of tobacco products will stop using such
    products,’” which “necessarily implies a comparative
    analysis.” Id. (quoting 21 U.S.C. § 387j(c)(4)(A)). 12 Thus, the
    statute and June 2019 Guidance are clear about comparative
    analysis. 13 Accordingly, “[b]ecause the [June] 2019 Guidance
    gave fair notice of the analysis the agency would perform and
    12
    Additionally, even if such express authority were
    lacking, the “FDA certainly has implied authority”—for the
    reasons the Court of Appeals for the Fifth Circuit explained—
    to consider comparative risk as it did here. Wages & White
    Lion, 41 F.4th at 435.
    13
    Liquid Labs also contends that it was misled because
    the June 2019 Guidance “focuses exclusively on the
    physiological health risks associated with the compared
    products, not behavioral impacts.” Reply Br. at 10-11; see also
    id. at 25. As the Wages & White Lion court explained,
    however, “[i]nitiation and cessation behaviors are
    physiological health risks.” 41 F.4th at 434 (emphasis
    omitted); see also Prohibition Juice, 45 F.4th at 19-20 (“The
    degree to which a harmful product entices and addicts new
    users is inarguably a component of the ‘health risk’ it poses.”).
    19
    the purpose of those comparisons, . . . the agency did not create
    unfair surprise by focusing on comparisons between otherwise
    similar flavored and nonflavored products.” 14 Prohibition
    Juice, 45 F.4th at 24. 15
    14
    Liquid Labs’ related argument that the FDA, in effect,
    engaged in the modified risk tobacco products inquiry or the
    required proof akin to that necessary for drugs is also
    unpersuasive. As discussed herein, the FDA did not impose
    “entirely different (and far more stringent) requirements,” Pet.
    Br at 52, than those contemplated by the governing statute, see
    also 
    21 U.S.C. §§ 355
    (b)(1)(A), 387k. “Moreover, the fact that
    the FDA has other authorities through which it can approve
    other products . . . does not release the FDA from following its
    statutory mandate here to approve only tobacco products the
    sale of which it determines ‘would be appropriate for the
    protection of the public health,’” and Liquid Labs has given us
    “no persuasive reason to think that those other authorities
    somehow limit the inquiry the FDA may make in reaching” its
    “determination.” Prohibition Juice, 45 F.4th at 20 (quoting 21
    U.S.C. § 387j(c)(2)); see also Gripum, LLC, 47 F.4th at 559.
    15
    Liquid Labs claims that it provided evidence aligning
    with the FDA’s June 2019 Guidance. Liquid Labs has not
    shown, however, that the FDA erred in concluding that the
    evidence was insufficient. Wages & White Lion, 41 F.4th at
    439; see also Breeze Smoke, 18 F.4th at 507 (“declin[ing] to
    embrace” petitioner’s claim “that the FDA’s willingness to
    consider some forms of evidence, explicitly phrased as such,
    required the FDA to accept that evidence as meeting a statutory
    requirement even where the FDA found the evidence
    unsatisfactory”).
    20
    For these reasons, the FDA did not apply unannounced
    or changed standards for PMTAs.
    3
    The FDA’s decision to decline to review Liquid Labs’
    marketing plan does not change the result because there is no
    indication the plan would have made up for the deficiencies the
    FDA identified in Liquid Labs’ applications. See Delaware
    Riverkeeper Network v. Sec’y Pa Dep’t of Env’t Prot., 
    833 F.3d 360
    , 377 (3d Cir. 2016) (“[M]istakes that have no bearing
    on the substantive decision of an agency do not prejudice a
    party.”); see also, e.g., Prohibition Juice, 45 F.4th at 25
    (concluding petitioner failed to show that the FDA’s failure to
    consider its marketing plan “could have changed the agency’s
    decision on their applications”); Wages & White Lion, 41 F.4th
    at 442 (concluding that even if the FDA inadequately reviewed
    petitioners’ marketing plans, the error was harmless because
    petitioners failed to “show that they would have received
    authorization had [the] FDA considered the[] plans”). 16 For
    16
    In a divided opinion, the Court of Appeals for the
    Eleventh Circuit remanded after concluding the FDA’s failure
    to review the petitioners’ marketing plans was both arbitrary
    and capricious and harmful. See Bidi Vapor LLC v. U.S. Food
    & Drug Admin., 
    47 F.4th 1191
     (11th Cir. 2022). Among other
    things, the Bidi Court distinguished Prohibition Juice on the
    grounds of “concessions . . . made . . . at oral argument” before
    the Court of Appeals for the District of Columbia Circuit. Id.
    at 1208. The purported concessions echoed the Prohibition
    Juice petitioners’ briefing, which did not “identify how they
    were harmed from the FDA’s failure to consider essentially the
    same [marketing] measures it had previously rejected.”
    21
    example, to address youth use, Liquid Labs’ marketing plan
    lists, among other things, “age verification measures,” a
    “mystery shopper” program, Pet. Br. at 12, and a prohibition
    on marketing material “that could be perceived to be targeting
    individuals below the legal vaping age,” JA 322-23, but these
    are similar, if not identical, to the kinds of approaches the FDA
    found did not address this serious problem, see, e.g., JA 89-91,
    125-27 (April 2020 Guidance); see also Prohibition Juice, 45
    F.4th at 25 (explaining that “self-verification of age at the point
    of sale and . . . less vibrant marketing unappealing to youth”
    “track measures the FDA in its 2020 guidance deemed
    inadequate”); Wages & White Lion, 41 F.4th at 442
    (explaining the “FDA had already explained,” for example,
    that “products . . . [being] only sold in age-gated vape and
    specialty tobacco shops and through age-gated online sales”
    “do not work”) (emphasis omitted). Liquid Labs has not
    explained how the approaches in its plan differ from ones
    previously found insufficient or how its marketing plans would
    have cured other noted deficiencies in its applications,
    particularly given the FDA’s earlier conclusion that “focusing
    on how the product was sold would not be sufficient to address
    youth use of [flavored cartridge-based] products.” 17 JA 125.
    Prohibition Juice, 45 F.4th at 25 (“In response to questioning .
    . . at oral argument, the manufactures again did not identify
    . . . .”). Liquid Labs also did not provide such an explanation,
    so it is like Prohibition Juice and thus different from Bidi.
    17
    Liquid Labs tried to distinguish its marketing plan
    from “many” other companies’, but it fails to explain how its
    purportedly distinctive features—e.g., selling via online third-
    party distributors rather than selling through its own website—
    make a meaningful difference or address the concerns about
    youth usage.
    22
    Because Liquid Labs has not shown that its marketing
    plans differ from those previously rejected or that its plans
    would have rectified the scientific deficiencies, the marketing
    plans would not change the result. Accordingly, even
    assuming the FDA erred in declining to review Liquid Labs’
    marketing plans, the error was harmless. See Shinseki v.
    Sanders, 
    556 U.S. 396
    , 411 (2009) (stating one of “the factors
    that inform[s] a reviewing court’s ‘harmless-error’
    determination” is “an estimation of the likelihood that the
    result would have been different”).
    4
    Contrary to Liquid Labs’ claim, the FDA did not ignore
    that Liquid Labs sought approval for “bottled e-liquids
    intended for use with open-systems devices.” Pet Br. at 45-46.
    The FDA acknowledged that “there may be differential appeal
    of certain product styles,” but pointed to evidence where “the
    removal of one flavored product option prompted youth to
    migrate to another ENDS type that offered the desired flavor
    options” as “underscoring the . . . role of flavor in driving
    appeal,” and explained, based on the evidence it reviewed, that
    “the role of flavor is consistent” “across . . . different device
    types.” JA 68-69.            Because these observations and
    conclusions are backed “with substantial evidence, . . . we have
    no basis to second-guess [them].” Prohibition Juice, 45 F.4th
    at 26; Wages & White Lion, 41 F.4th at 437-38. We therefore
    reject Liquid Labs’ argument that the FDA ignored the specific
    characteristics of its products.
    Finally, Liquid Labs argues that the FDA erred by
    failing to consider all segments of the population, including
    23
    adults currently using flavored ENDS who may lose their
    ability to access them and who may turn to the “illicit market”
    as a result. Pet. Br. at 51. 18
    Liquid Labs’ argument lacks merit. As an initial matter,
    all non-tobacco and/or menthol flavored ENDS are not banned,
    and thus, there is no reason for the FDA to examine Liquid
    Labs’ speculative “entire[] eliminat[ion]” claim. Pet Br. at 50.
    In any event, the FDA explained that: (1) the relevant statutory
    authorities required it to account for “the risks and benefits to
    the population as a whole,” JA 64 n.ii; (2) its “review [wa]s
    focused on the risk to youth nonusers as well as the potential
    benefit to adult smokers as current users, as they are the group
    through which the potential benefit to public health is most
    substantial and could overcome the known risk to youth,” JA
    64 n.ii; see JA 65 n.vii; and (3) it considered, among other
    things, (a) “the likelihood that an authorization will increase or
    decrease the number of tobacco users in the overall
    population,” JA 65 n.vii, (b) “the degree of benefit to a flavored
    ENDS product over a tobacco-flavored variety in facilitating
    18
    Liquid Labs also argues that the FDA did not consider
    “less disruptive alternatives” to issuing the denial order. Pet.
    Br. at 51 (quoting Wages & White Lion, 16 F.4th at 1139). The
    FDA, however, “was not required to consider alternative
    regulatory approaches before denying the manufacturers’
    applications for premarket approval.” Prohibition Juice, 45
    F.4th at 26. Further, the portion of the Wages & White Lion
    opinion that Liquid Labs cites discusses alternatives in the
    context of its conclusion that the FDA “chang[ed] from its no-
    long-term-studies-necessary policy to its apparent long-term-
    studies-required policy.” 16 F.4th at 1139. We have already
    concluded, however, that no such change occurred.
    24
    smokers completely switching or significantly reducing their
    smoking,” JA 72, and (c) the lack of conclusive evidence
    “regarding the role of flavors in prompting switching among
    adult smokers,” JA 72; see also JA 64 n.vi (“[I]n the absence
    of strong evidence generated by directly observing the
    behavioral impacts of using a flavored product vs. a tobacco-
    flavored product over time, we are unable to reach a conclusion
    that the benefit outweighs the clear risks to youth.”). Thus, the
    record shows that the FDA considered whether Liquid Labs’
    products were appropriate for the protection of public health
    from several vantage points, provided evidence for its
    particular focus, and concluded that the evidence Liquid Labs
    submitted came up short.
    B
    In light of the foregoing, Liquid Labs’ additional
    arguments lack merit. The FDA reviewed Liquid Labs’
    application in conformity with, among other things, its
    statutory authorities and publicly issued guidance, and thus did
    not act ultra vires. Likewise, because the FDA did not adopt a
    “new secret standard” or otherwise change course, it also did
    not violate the Act or APA and had no obligation to proceed
    through notice-and-comment rulemaking.
    III
    For the foregoing reasons, we will deny the Petition for
    Review.
    25