记录一些平时接触到的存储知识。由于是笔记而非教程，因此内容不求连贯，有基础的同学可作查漏补缺之用。
Fig. 12 Left: HDDs as a JBOD, present to OS as 12 independent devices (sd*), running a Ceph OSD service on each device. Right: speedup performance with high-end RAID cards.
水平及维护精力所限，文中不免存在错误或过时之处，请酌情参考。 传播知识，尊重劳动，年满十八周岁，转载请注明出处。
解决办法：RAID、JBOD、LVM 等等。
2 容量不够，JBOD (Just a Bunch Of Disks) 来凑 2.1 定义JBOD 在 Wikipedia 中没有单独的词条， 而是归类在 Non-RAID drive architectures 中。
JBOD 是一种架构，
最简化的理解：使用 JBOD 模式，那机器上插了几个盘，操作系统中就能看到几个 /dev/sd* 设备。
比如下图是一台 12 盘的 Ceph 机器。Ceph 的设计中，每个盘由一个独立的进程来管理，也就是它的 OSD 进程， 所以就适合做 JBOD（但 RAID 也是可以的，右边所示 [2]），
Fig. 12 Left: HDDs as a JBOD, present to OS as 12 independent devices (sd*), running a Ceph OSD service on each device. Right: speedup performance with high-end RAID cards.
2.2 优缺点JBOD 是硬件特性，主板的存储控制器自带这个功能，一般的 RAID 卡也支持 JBOD 模式。
也有一些具有类似功能的软件，比如 LVM (Logical Volume Manager)。 下一篇再介绍。
3 花钱办事：硬件 RAID 卡数据冗余+提升性能 3.1 定义RAID 是 Redundant Array of Independent Disks 的缩写，独立磁盘冗余阵列，可以提供多种级别的数据容易，防止因为单个磁盘故障导致数据丢失或不可用。 RAID 本身只是一种技术。实现上可以是硬件 RAID 卡，也可以是纯软件方案。
我们接下来讨论的主要是硬件 RAID 卡。
3.2 分类 3.2.1 按 RAID 模式分类可参考 [2]，不错的介绍和软件 raid 教程。
3.2.2 按有无缓存（write back cache）分类RAID 卡上有没有内存：
以下是 Broadcom MegaRAID 9560-16i 8GB RAID 卡，自带 8C 处理器，8GB 内存。
Fig. Broadcom MegaRAID 9560-16i 8GB RAID Controller.
RAID 卡本身作为 PCIe 卡插到主板上，磁盘通过 SATA 接口插到右侧（也可以加转换线，将 PCIe 接口的 NVME SSD 插到右侧）。 一些产品参数 [3]：
NVME SSD 有两种常见的接口格式：
如果以上速度还不满足业务需求，可以考虑加上 RAID 卡，下面是 M.2 格式的多个 NVME SSD 做成 RAID 的样子：
Fig. Hardware RAID10 over NVME SSDs. Image Source
前面 Broadcom 那个卡也支持 NVME RAID，但支持的 PCIe 格式的 NVME，而且需要通过 PCIe 扩展线来连接。
3.4 RAID 卡上为什么要配备电池（或超级电容）？ 3.4.1 突然掉电的问题对于有 WB cache 的，如果数据写到了 cache，但还没写到磁盘，掉电了怎么办？会导致数据丢失。 所以引入了配套的电池（BBU, Battery Backup Unit），
电池能解决掉电丢数据问题，但寿命和故障率是个问题。近几年新出来的另一种保持数据的方式是超级电容（supercapacitors）。
BBU or SuperCapacitor [4]:
再回到 RAID 卡本身。东西好是好，但贵，有没有降本的方案呢？
3.5.1 VROC (Virtual Raid On CPU)Intel CPU 独有的技术，CPU 内置硬件模块，官方介绍。
没用过。
参考资料记录一些平时接触到的存储知识。由于是笔记而非教程，因此内容不求连贯，有基础的同学可作查漏补缺之用。
Fig. LVM concepts, and how userspace file operations traverse the Linux storage stack.
水平及维护精力所限，文中不免存在错误或过时之处，请酌情参考。 传播知识，尊重劳动，年满十八周岁，转载请注明出处。
Device mapper（设备映射器） 是 Linux 内核提供的一个框架，用于将物理块设备（physical block devices） 映射到更上层的虚拟块设备（virtual block devices）。
Fig. Device Mapper 在 Linux 存储栈中的位置（图中间部分）
1.3 使用场景及典型应用dm-cache：组合使用 SSD 和 HDD 的混合卷（hybrid volume）
A hybrid volume is any volume that intentionally and opaquely makes use of two separate physical volumes. For instance, a workload may consist of random seeks so an SSD may be used to permanently store frequently used or recently written data, while using higher-capacity rotational magnetic media for long-term storage of rarely needed data. On Linux, bcache or dm-cache may be used for this purpose.
Logical Volume Manager （LVM，逻辑卷管理器）1998 年引入内核，是一个基于 device mapper 的框架， 为内核提供逻辑卷管理能力。
LVM 可以认为是物理磁盘和分区之上的一个很薄的软件层， 能方便换盘、重新分区和备份等等管理工作。
2.2 LVM 中的概念/术语图解Fig. LVM concepts, and how userspace file operations traverse the Linux storage stack.
2.3 使用场景LVM 使用场景：
常规的文件系统，例如 XFS、EXT4 等等，都不能跨多个块设备（device）。 也就是说，创建一个文件系统时，只能指定一个特定的 device，比如 /dev/sda。
要跨多个盘，只能通过 RAID、JBOD、LVM 等等技术将这些块设备合并成一个逻辑卷， 然后在这个逻辑卷上初始化文件系统。
3.2 Cross-device 文件系统更高级一些的文件系统，是能够跨多个块设备的，包括，
上一节已经介绍到，在块设备上初始化文件系统，就可以创建文件和目录了。 这里所说的块设备 —— 不管是物理设备，还是逻辑设备 —— 穿透之后终归是一个插在本机上硬件设备。
有了虚拟化之后，情况就不一样了。 比如有一类特殊的 Linux 设备，它们对操作系统呈现的确实是一个块设备， 但其实底层对接的远端存储系统，而不是本机硬件设备。
在云计算中，这种存储类型称为“块存储”。
4.1 典型块存储产品块存储（Block Storage），也称为 block-level storage，是公有云和私有云上都非常常见的一种存储。 各家的叫法或产品名字可能不同，例如，
块存储工作在块级别（device-level），可以直接访问数据并实现高性能I/O。 因此它提供高性能、低延迟和快速数据传输。
4.3 使用场景和使用方式使用场景：
使用方式：
将这个块挂载到想使用的机器上，这时呈现给这台机器的操作系统的是一个块设备（/dev/xxx），
Storage Decision. Image Source
AWS 对文件存储、对象存储和块存储有一个不错的介绍文档。 其中提到的块存储的设计：
创建一个 RBD 块设备的大致步骤：
$ ceph osd pool create {pool-name} [{pg-num} [{pgp-num}]] [replicated] \ [crush-rule-name] [expected-num-objects] $ rbd pool init {pool-name} $ rbd create --size {size MB} {pool-name}/{image-name} 4.5.2 RBD 的后端存储：Ceph 对象存储Ceph 的设计比较特殊，同时支持三种存储类型：
块存储（block storage），类似 AWS EBS。
背后，每个块存储中的 “block”（4.4 节中介绍的 block 概念）， 实际上最后是一个 Ceph 对象存储中的 object。 也就是 Ceph 的块存储是基于 Ceph 的对象存储。
Fig. Ceph RBD IO. Each object is fix-sized, e.g. 4MB by default. Image Source
4.5.4 客户端代码实现两种使用方式，二选一：
Fig. Ceph RBD workflow. Image Source
How Are We Innovating with Secure Secrets Rotation Techniques? With the rapid expansion of digitized environments, the demand for effective and secure identity management has surged. Organizations are increasingly relying on machine identities or Non-Human Identities (NHIs) to safeguard their data and ensure smooth operations. However, how are we, as data management experts, innovating secure […]
The post Innovating with Secure Secrets Rotation Techniques appeared first on Entro.
The post Innovating with Secure Secrets Rotation Techniques appeared first on Security Boulevard.
Why is Proactive Security Crucial in IAM? Have you ever weighed the impact of security breaches and data leaks on your business? Increasingly, organizations are finding tremendous value in adopting a proactive security approach, particularly in the realm of Identity and Access Management (IAM). This is the first and often most crucial line of defence […]
The post Proactive Approaches to Identity and Access Management appeared first on Entro.
The post Proactive Approaches to Identity and Access Management appeared first on Security Boulevard.
Why is Cloud Security Imperative for Asset Protection? As businesses increasingly migrate their operations to the cloud, the demand for effective cloud security strategies gains precedence. The criticality of this requirement becomes glaringly obvious when one considers asset protection. But how does cloud security play into the grand scheme of asset protection? And how does […]
The post Navigating Cloud Security for Future Proofing Your Assets appeared first on Entro.
The post Navigating Cloud Security for Future Proofing Your Assets appeared first on Security Boulevard.
Why is Secrets Vaulting Crucial in Today’s Cybersecurity Landscape? In a world increasingly dependent on cloud-based services, how do organizations ensure maximum security while maintaining operational efficiency? The answer might just lie in an under-explored area of cybersecurity: Non-Human Identities (NHIs) and secrets management. A Deeper Dive into Non-Human Identities and Secrets Vaulting NHIs are […]
The post Gaining Confidence Through Effective Secrets Vaulting appeared first on Entro.
The post Gaining Confidence Through Effective Secrets Vaulting appeared first on Security Boulevard.
A better path forward for cybersecurity
Why is it that cybersecurity is struggling to keep pace with the rapidly evolving threat landscape? We spend more and more, tighten our perimeters, and still there are trillions of dollars being lost to cybercrime and cyber attacks. Setting aside the direct costs to individuals and businesses, and the indirect costs we all experience such as a lack of trust in our banks and other institutions — what about the role of these trillions of dollars in growing an ever-more sophisticated criminal economy?
With that as context, I’ve been thinking a lot about whether there is something that we need to do in cybersecurity to shift the way we build, deliver, and consume solutions. Yes, at DeepTempo we see a world in which an ever-improving Collective Defense is possible thanks to the emergence of Foundation LogLMs. But is using AI to sure up the shaky foundations of cybersecurity enough? What if the security industry itself is structured in such a way that it hampers the innovations we need?
Why Cybersecurity Must Shift Towards Horizontal Competition IntroductionThe cybersecurity industry is dominated by vertically integrated giants. In 2024, Palo Alto Networks emphasized their role as a platform provider in their conversations with investors and the market — “platformization” it calls the strategy. Platform firms attempt to deliver everything — from log collection to threat detection — locking customers into rigid ecosystems. While such an approach makes sense for Palo Alto as it attempts to diversify away from its fading firewall products — is “platformization” best for the future of the cybersecurity industry?
A shift toward horizontal competition, where specialized players excel within layers of the stack and are able to grow into large and valuable firms without being subsumed into the platform companies, offers hope.
Why Horizontal Competition IS EmergingThere are signs of a more horizontal structure emerging in cyber security. Here are a few drivers powering this transition:
Cloud Services Data Layers
Cloud platforms like AWS, Azure and GCP have transformed the technology industry. And Snowflake, built upon these infrastructure cloud providers, has delivered an easy-to-use and massively scalable datalake. Enterprises now see previously daunting capabilities like pulling data into a central location and transforming it for their use as readily available. And they’ve invested in enterprise data layers — and are looking for additional use cases, such as cyber security, to drive the ROI on these investments.
Cloud Services and Cloud Native Architectures
Maybe even more importantly, the rise of cloud services led to cloud-native architectures. We now have a generation of architects who expect to be able to compose loosely coupled components into a preferred overall solution. They understand how to build environments that assume failure is inevitable — and as a result are extremely resilient. What they must think when they see the tightly coupled, brittle architectures of security today!
LLM based UX
LLMs like ChatGPT are rewriting the rulebook for how we interact with complex systems. Need to query logs or explore anomalies? You don’t need a PhD in cybersecurity — natural language UX bridges the gap. As these tools improve, the grip of vertically integrated SaaS providers loosens. This is a much broader phenomenon than security; some investors have suggested that all verticalized SaaS is vulnerable.
Cost Savings Through Shared Data Platforms
Enterprises today demand efficiency. Why store duplicate data for cybersecurity when platforms like Snowflake can do double duty for analytics and fraud detection? This shared infrastructure, optimized for multiple use cases, encourages specialization while cutting costs.
The DevOps ethos of team autonomy
A central tenant of the DevOps ethos is that tools should not be forced upon a team; teams need the autonomy to select their operating environment. After all, they live with the results. By contrast, in many traditional sectors, including much of security, solutions are often sold top-down with less input from the operators.
The Status Quo’s Inadequacy Against Rising Threats
The bad guys are getting smarter — AI is helping them move faster, hit harder, and hide better. Stagnant, vertically integrated systems can’t keep up. Horizontal approaches encourage adaptability, diversity, and innovation, which we desperately need to outpace attackers.
A market in which better approaches have a great shot at achieving scale — and they need not bet upon a particular platform to get to market — is a more enjoyable market. And a more innovative one. Here are a few specific reasons that I am motivated by helping to bring more openness, and horizontal competition, to cyber security:
We are in Crisis
Cybersecurity isn’t just about protecting companies — it’s about safeguarding society. A breach in one firm can cascade into a crisis for thousands. Horizontal competition fosters innovation and ensures that cutting-edge defenses are broadly accessible. And we must try something new — the old ways of doing business in cybersecurity are not working.
Lock-in is Bad — and Private Equity is Coming for Your Budget
The history of technology includes technology companies forming closer and closer ties to their customers — and then slowing down their innovation and becoming cash cows, often purchased by private equity and intent on extracting as much as possible from their users. Broadcom, of course, behaves this way; however, we can also see more than a dozen purchases of cybersecurity assets by Private Equity in the last years.
Encouraging a Resilient Ecosystem
Homogeneity breeds vulnerability. A world reliant on just a few vertically integrated providers creates single points of failure attackers can exploit. Horizontal competition spreads the load, promoting a resilient, diverse ecosystem that’s harder to compromise.
There is much to do if we want to shift cybersecurity towards one in which best-of-class horizontal solutions succeed. Here are a few specific recommendations:
Recommendations for Clouds and Stakeholders
The cybersecurity market is enormous — already well more than $200bn per year. Clouds and platforms like Snowflake have a central role to play in transforming cybersecurity. My recommendation to them is essentially — Amp It Up! You understand the sheer size and criticality of data in cyber security. Also — Snowflake’s ability to run NativeApps is hugely differentiating. Snowflake’s capabilities include enabling security teams to very quickly try or POC interesting technologies, further reducing the opportunity of legacy security to keep up.
Vision for the Future
Imagine a cybersecurity industry in which we all learn from each other — what is normal, and what is problematic, and share that intelligence via adaptive deep learning base systems as opposed to outdated signatures. Such an industry will return the initiative to the defenders. We have the opportunity to play an important part in saving trillions of dollars of losses and improving trust in our institutions.
The move toward horizontal competition isn’t inevitable — however, it seems like the only path forward. As attackers grow bolder and tools like AI raises the stakes, the status quo simply won’t cut it. By embracing openness, collaboration, and innovation, we can create a cybersecurity industry that protects not just companies but the world itself.
Platforms are the Problem was originally published in DeepTempo on Medium, where people are continuing the conversation by highlighting and responding to this story.
The post Platforms are the Problem appeared first on Security Boulevard.