NASNet

AI领域的“自动建筑师”：深入浅出NASNet

想象一下，如果你想盖房子，传统方式是请建筑师根据经验和知识，手工绘制一张张详细的图纸，包括房间布局、楼层结构、供水供电系统等等。这需要建筑师拥有多年的专业知识和丰富才能。而如果在人工智能（AI）领域，设计一个像神经网络这样的“智能建筑”，其复杂程度可能比盖房子还要高得多！

长久以来，构建高性能的神经网络模型都是AI研究人员和工程师的专属“绝活”。他们需要凭借深厚的理论知识和反复的实验，小心翼翼地挑选合适的网络层（例如卷积层、全连接层），巧妙地设计层与层之间的连接方式（比如跳过连接、残差连接），并确定每一层的具体参数（如卷积核大小、滤波器数量）。这个过程不仅耗时耗力，而且对AI专家的经验要求极高，就像手艺精湛的老木匠一锤一凿地打造精致家具一样。然而，人类的精力总是有限，面对海量的可能性，我们很难确保找到那个“完美”的设计。

正是在这样的背景下，一个被称为“神经架构搜索”（Neural Architecture Search, 简称NAS）的革命性概念应运而生。它就像一位拥有无限精力和创造力的“自动建筑师”，能够自动探索并设计出高性能的神经网络结构。而NASNet，正是这个“自动建筑师”设计出的众多优秀“作品”中的一个里程碑式的代表。

什么是神经架构搜索（NAS）：AI自己设计AI

要理解NASNet，我们首先得认识它的“幕后推手”——神经架构搜索（NAS）。简单来说，NAS就是一套算法，让AI自己去设计和优化AI模型，从而极大地拓展了模型设计的可能性。这个过程可以形象地比喻成请来一个“机器人大厨”，它不再依赖人类大厨的菜谱，而是能自己尝试各种食材（神经网络的各种操作单元如卷积、池化），搭配不同的烹饪方法（连接方式），然后品尝（评估性能）自己做出的菜肴，并根据“口味”（模型在特定任务上的表现）持续改进，最终找到一道道美味无比的菜品（高性能的神经网络架构）。

NAS“机器人大厨”工作的核心要素有三个:

  1. 搜索空间（The “食材仓库”）: 这定义了“机器人大厨”可以使用哪些基础食材以及食材之间的组合规则。NASNet的创新之处在于，它没有试图一次性设计整个复杂的“盛宴”，而是专注于设计可重复使用的“菜肴模块”——称为“单元”（Cell），然后将这些单元像搭乐高积木一样组合起来。这大大缩小了搜索范围，让问题变得更容易解决。
  2. 搜索策略（The “烹饪方法”）: 这是“机器人大厨”如何探索“食材仓库”以寻找最佳组合的策略。NASNet最初采用了强化学习（Reinforcement Learning）作为其核心策略。你可以想象有一个“控制大脑”（通常是一个循环神经网络RNN），它会根据过去的经验“预测”出一套新的“菜品组合”（生成一个神经网络架构），然后让它去“烹饪”（训练这个架构），“品尝”（评估性能），最后根据“品尝结果”来调整下一次“预测”的方向，力求做得更好。除了强化学习，还有贝叶斯优化、进化算法、基于梯度的方法等多种“烹饪方法”可供选择。
  3. 性能评估策略（The “品尝师”）: 每当“机器人大厨”做出一道新菜，就需要“品尝师”来打分。在AI中，就是通过在验证集上测试模型的准确率或效率来打分。这是整个过程中最耗费时间和计算资源的部分，因为每个被提议的架构都需要经过训练和评估。

NASNet：由AI自己设计出的“明星架构”

NASNet并不是一套搜索算法，而是一套由NAS搜索算法发现并验证过的神经网络架构。它是由谷歌大脑团队在2017年提出的，旨在解决图像识别领域的挑战。

NASNet最关键的贡献在于它通过NAS发现了一系列性能卓越的可迁移卷积单元。就像“机器人大厨”没有直接设计完整的宴席，而是先设计出了两种最核心、最好用的“菜肴模块”：

  • 普通单元（Normal Cell）: 这种单元的主要功能是提取图像特征，但不会改变图像特征图的空间大小，就像一道菜，虽然口味变得更丰富，但分量没有变。
  • 归约单元（Reduction Cell）: 这种单元能有效地减少图像特征图的空间分辨率，就像把一道大菜浓缩成精华，同时保持其营养和风味，这有助于网络更有效地捕捉大范围的特征，并降低计算量。

然后，研究人员或者更进一步地，由NAS算法将这些“普通单元”和“归约单元”以特定的方式堆叠起来，就形成了完整的NASNet网络架构。这种模块化的设计使得在小数据集上（例如CIFAR-10）搜索到的优秀单元结构，可以非常高效地迁移到大型数据集（例如ImageNet）上，并获得同样出色的表现，甚至超越了之前人类专家手工设计的最佳模型。

NASNet的出现，在图像分类任务中取得了当时最先进的准确率，例如NASNet-A在ImageNet上达到了82.7%的top-1准确率，比人类设计的最优架构提高了1.2%。它还有NASNet-B和NASNet-C等变体，展示了这种自动化设计方法的强大能力。

NASNet的优势：AI的超能力

NASNet以及它所代表的NAS技术，带来了多方面的显著优势：

  • 超越人类的性能: NAS可以发现人类专家难以想象或发现的优秀架构，在特定任务上经常能超越人类手工设计的模型，正如NASNet在图像识别领域的突出表现。
  • 自动化与高效: 大大减少了AI专家手动设计和调试神经网络结构的时间与精力，将AI模型设计的门槛降低，使得更多人可以利用高性能的AI模型。
  • 可移植性: 通过搜索通用单元或模块，可以在一个任务或数据集上学习到的结构，迁移到其他任务或数据集上，并保持优异性能，这正是NASNet的核心贡献之一。
  • 广泛应用: NASNet等由NAS寻找到的模型不仅在图像分类等任务上表现出色，还在目标检测、图像分割等计算机视觉任务中取得了优于人工设计网络的性能。

挑战与未来方向：持续进化的“自动建筑师”

尽管NASNet带来了巨大的突破，但神经架构搜索仍然面临一些挑战：

  • 巨大的计算成本: 这是NAS最大的“痛点”。早期的NAS方法可能需要成千上万个GPU天才能完成搜索，这笔“电费”可不是小数目。即便NASNet通过搜索单元结构已将训练时间加速了7倍以上，但依然需要大量的计算资源。
    • 改进方向: 为解决这一问题，研究人员正在探索更高效的搜索算法，例如基于梯度的方法、一次性（one-shot）NAS、多重保真度（multi-fidelity）方法，以及通过权重共享、减少训练周期、使用代理模型或在小数据集上预搜索等技术来加速评估过程。例如，最新的进展包括使用“差分模型缩放”来更有效地优化网络的宽度和深度。
  • 模型可解释性: 自动生成的复杂架构，有时像一个“黑盒子”，我们难以完全理解其内部工作原理，这可能会影响模型的可靠性和可信度。
  • 搜索空间的设计: 搜索空间的设计质量直接影响到最终结果的好坏，如何设计更智能、更合理的搜索空间仍是研究重点。

NAS是AutoML（自动化机器学习）领域的重要组成部分，未来的研究方向将继续探索更高效的搜索算法、更智能的搜索空间，以及提高NAS的可解释性，让“自动建筑师”不仅能盖出好房子，还能解释清楚为什么这样盖最好。

总结

NASNet的出现，标志着AI领域从“人类设计AI”向“AI设计AI”迈出了重要一步。它不仅在图像识别等任务上取得了令人瞩目的成就，更重要的是，它验证了神经架构搜索（NAS）的巨大潜力。虽然NAS技术仍面临计算成本高昂等挑战，但科学家们正不断努力，使其变得更加高效、智能和易于理解。在未来，我们可以期待AI这位“自动建筑师”设计出更多意想不到、性能更卓越的智能“建筑”，推动人工智能在各个领域实现新的突破。

NASNet

The AI Architect: An In-Depth Look at NASNet

Imagine if you wanted to build a house. The traditional way is to hire an architect to draw detailed blueprints by hand based on experience and knowledge, including room layout, floor structure, water and power supply systems, and so on. This requires the architect to have years of professional knowledge and talent. But if you were to design an “intelligent building” like a neural network in the field of Artificial Intelligence (AI), the complexity might be much higher than building a house!

For a long time, building high-performance neural network models has been the exclusive “knack” of AI researchers and engineers. They need to rely on profound theoretical knowledge and repeated experiments to carefully select appropriate network layers (such as convolutional layers, fully connected layers), cleverly design the connections between layers (such as skip connections, residual connections), and determine the specific parameters of each layer (such as kernel size, number of filters). This process is not only time-consuming and laborious but also requires extremely high experience from AI experts, just like a skilled carpenter crafting exquisite furniture with a hammer and chisel. However, human energy is always limited. Faced with massive possibilities, it is difficult for us to ensure finding that “perfect” design.

Against this background, a revolutionary concept called “Neural Architecture Search“ (NAS) came into being. It is like an “automated architect” with unlimited energy and creativity, capable of automatically exploring and designing high-performance neural network structures. NASNet is a milestone representative among the many excellent “works” designed by this “automated architect”.

What is Neural Architecture Search (NAS): AI Designing AI

To understand NASNet, we first need to understand its “driving force”—Neural Architecture Search (NAS). Simply put, NAS is a set of algorithms that allows AI to design and optimize AI models by itself, thereby greatly expanding the possibilities of model design. This process can be vividly compared to hiring a “robot chef.” It no longer relies on the recipes of human chefs but can try various ingredients (various operation units of neural networks such as convolution, pooling) by itself, match different cooking methods (connection methods), then taste (evaluate performance) the dishes it makes, and continuously improve based on “taste” (model performance on specific tasks), finally finding delicious dishes (high-performance neural network architectures).

The core elements of the NAS “robot chef’s” work are three:

  1. Search Space (The “Ingredient Warehouse”): This defines which basic ingredients the “robot chef” can use and the combination rules between ingredients. The innovation of NASNet is that it didn’t try to design the entire complex “feast” at once, but focused on designing reusable “dish modules”—called “Cells”—and then assembling these cells like Lego blocks. This greatly narrows the search scope and makes the problem easier to solve.
  2. Search Strategy (The “Cooking Method”): This is the strategy of how the “robot chef” explores the “ingredient warehouse” to find the best combination. NASNet initially adopted Reinforcement Learning as its core strategy. You can imagine there is a “controlling brain” (usually a Recurrent Neural Network, RNN), which will “predict” a new set of “dish combinations” (generate a neural network architecture) based on past experience, then let it “cook” (train this architecture), “taste” (evaluate performance), and finally adjust the direction of the next “prediction” based on the “tasting result,” striving to do better. Besides reinforcement learning, there are various “cooking methods” such as Bayesian optimization, evolutionary algorithms, and gradient-based methods available.
  3. Performance Evaluation Strategy (The “Taster”): Whenever the “robot chef” makes a new dish, a “taster” is needed to score it. In AI, this is scoring by testing the accuracy or efficiency of the model on a validation set. This is the most time-consuming and computationally expensive part of the entire process because every proposed architecture needs to be trained and evaluated.

NASNet: A “Star Architecture” Designed by AI Itself

NASNet is not a search algorithm, but a set of neural network architectures discovered and verified by the NAS search algorithm. It was proposed by the Google Brain team in 2017 to address challenges in the field of image recognition.

The most critical contribution of NASNet is that it discovered a series of high-performance transferable convolutional cells through NAS. Just like the “robot chef” didn’t directly design a complete banquet, but first designed two of the most core and useful “dish modules”:

  • Normal Cell: The main function of this cell is to extract image features, but it will not change the spatial size of the image feature map, just like a dish, although the taste becomes richer, the portion remains unchanged.
  • Reduction Cell: This cell can effectively reduce the spatial resolution of the image feature map, just like concentrating a large dish into an essence while maintaining its nutrition and flavor, which helps the network capture large-scale features more effectively and reduce computation.

Then, researchers, or further, the NAS algorithm itself, stack these “Normal Cells” and “Reduction Cells” in a specific way to form the complete NASNet network architecture. This modular design allows excellent cell structures searched on small datasets (such as CIFAR-10) to be very efficiently transferred to large datasets (such as ImageNet) and achieve equally outstanding performance, even surpassing the best models manually designed by human experts before.

The appearance of NASNet achieved the most advanced accuracy at the time in image classification tasks. For example, NASNet-A achieved 82.7% top-1 accuracy on ImageNet, 1.2% higher than the optimal architecture designed by humans. It also has variants like NASNet-B and NASNet-C, demonstrating the powerful capability of this automated design method.

Advantages of NASNet: AI’s Superpower

NASNet and the NAS technology it represents bring significant advantages in many aspects:

  • Super-human Performance: NAS can discover excellent architectures that human experts can hardly imagine or discover, often surpassing human-crafted models on specific tasks, just like the outstanding performance of NASNet in the field of image recognition.
  • Automation and Efficiency: It greatly reduces the time and energy for AI experts to manually design and debug neural network structures, lowering the threshold for AI model design and allowing more people to utilize high-performance AI models.
  • Transferability: By searching for general cells or modules, structures learned on one task or dataset can be transferred to other tasks or datasets while maintaining excellent performance, which is one of the core contributions of NASNet.
  • Wide Application: Models found by NAS like NASNet not only perform well in tasks such as image classification but also achieve better performance than manually designed networks in computer vision tasks such as object detection and image segmentation.

Challenges and Future Directions: The Continuously Evolving “Automated Architect”

Although NASNet has brought huge breakthroughs, Neural Architecture Search still faces some challenges:

  • Huge Computational Cost: This is the biggest “pain point” of NAS. Early NAS methods might require thousands of GPU days to complete a search, and this “electricity bill” is not a small amount. Even though NASNet has accelerated training time by more than 7 times by searching for cell structures, it still requires a large amount of computing resources.
    • Improvement Directions: To solve this problem, researchers are exploring more efficient search algorithms, such as gradient-based methods, one-shot NAS, multi-fidelity methods, and techniques like weight sharing, reducing training epochs, using proxy models, or pre-searching on small datasets to accelerate the evaluation process. For example, recent progress includes using “differentiable model scaling” to optimize network width and depth more effectively.
  • Model Interpretability: Automatically generated complex architectures are sometimes like a “black box,” and we can hardly fully understand their internal working principles, which may affect the reliability and credibility of the model.
  • Design of Search Space: The quality of search space design directly affects the quality of the final result. How to design a more unexpected and reasonable search space remains a research focus.

NAS is an important part of the AutoML (Automated Machine Learning) field. Future research directions will continue to explore more efficient search algorithms, smarter search spaces, and improve the interpretability of NAS, so that the “automated architect” can not only build good houses but also clearly explain why building them this way is the best.

Summary

The emergence of NASNet marks an important step for the AI field from “human designing AI” to “AI designing AI”. It has not only achieved remarkable achievements in tasks such as image recognition, but more importantly, it verified the huge potential of Neural Architecture Search (NAS). Although NAS technology still faces challenges such as high computational costs, scientists are constantly working hard to make it more efficient, intelligent, and easy to understand. In the future, we can look forward to AI, the “automated architect,” designing more unexpected and superior intelligent “buildings,” promoting Artificial Intelligence to achieve new breakthroughs in various fields.